Re: Mysql/Innodb bug
Joe, this is a serious bug, because it can also spoil recovery from a backup. What MySQL version and Linux kernel you are running? Do you use RAID or NFS? How much RAM, how much swap partition of Linux? What do the corrupt binlog file names look like? Please show us all binlog files names, and the contents your binlog .index file. Did InnoDB recover from the crash without problems? Below we see that the crash happened in the memory allocator of MySQL. The problem might be a memory overrun or overwrite of the binlog file name. I will test running two ibbackups concurrently today. I doubt that ibbackup has to do with the corruption, unless this is yet another Linux file i/o bug. The binlog files are totally independent from ibbackup, which does not look at them at all. Best regards, Heikki Innobase Oy - Original Message - From: Joe Shear [EMAIL PROTECTED] Newsgroups: mailing.database.mysql Sent: Tuesday, September 24, 2002 12:56 AM Subject: Mysql/Innodb bug Appears to be a database memory corruption problem. The binlog filename had been trashed. Eventually, mysql/innodb blew up trying to do a malloc. We were accidentally running two copies of ibbackup at the same time at around 6pm on 9/21 and 9/22. We were accidentally running a third copy at around 11am on 9/21 and 9/22. The 11am backups appear to have succeeded. One of the 6pm backups reported success on each of 9/21 and 9/22. The binlog index file recorded 2 trashed binlog filenames. A binlog file was created on disk with a trashed filename. The trashed binlog filename has a time-last-modified of 5:05. At first glance it would seem that the fact we were running multiple copies of the backup software was unreleated to the core dump. Stack trace looks like: 0x806eea4 0x82d9218 0x83016a7 0x8301261 0x82c0b8e 0x806bf9d 0x806bc0e 0x808748c 0x809d845 0x8076eee 0x8079e3c 0x8074f14 0x80742c7 0x806eea4 handle_segfault__Fi + 428 0x82d9218 pthread_sighandler + 184 0x83016a7 chunk_alloc + 759 0x8301261 malloc + 209 0x82c0b8e my_malloc + 30 0x806bf9d get_lock_data__FP3THDPP8st_tableUibT1 + 113 0x806bc0e mysql_lock_tables__FP3THDPP8st_tableUi + 574 0x808748c open_ltable__FP3THDP13st_table_list13thr_lock_type + 280 0x809d845 mysql_update__FP3THDP13st_table_listRt4List1Z4ItemT2P4ItemUl15enum_duplicate s13thr_lock_type + 53 0x8076eee mysql_execute_command__Fv + 5322 0x8079e3c mysql_parse__FP3THDPcUi + 216 0x8074f14 do_command__FP3THD + 1460 0x80742c7 handle_one_connection__FPv + 655 .err file looks like 020920 10:26:33 mysqld ended 020920 10:32:20 mysqld started 020920 10:32:26 InnoDB: Log file /dblog/email/ib_logfile0 did not exist: new to be created InnoDB: Setting log file /dblog/email/ib_logfile0 size to 48 MB InnoDB: Database physically writes the file full: wait... 020920 10:32:30 InnoDB: Log file /dblog/email/ib_logfile1 did not exist: new to be created InnoDB: Setting log file /dblog/email/ib_logfile1 size to 48 MB InnoDB: Database physically writes the file full: wait... 020920 10:32:36 InnoDB: Log file /dblog/email/ib_logfile2 did not exist: new to be created InnoDB: Setting log file /dblog/email/ib_logfile2 size to 48 MB InnoDB: Database physically writes the file full: wait... 020920 10:32:42 InnoDB: Database was not shut down normally. InnoDB: Starting recovery from log files... InnoDB: Starting log scan based on checkpoint at InnoDB: log sequence number 0 3061070860 InnoDB: Doing recovery: scanned up to log sequence number 0 3061070860 InnoDB: Last MySQL binlog file position 0 5163, file name /data/email/mysql_binlog/binlog.927 020920 10:32:44 InnoDB: Flushing modified pages from the buffer pool... 020920 10:32:44 InnoDB: Started /usr/sbin/mysqld-max: ready for connections mysqld got signal 11; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked agaist is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail key_buffer_size=67104768 record_buffer=1044480 sort_buffer=1048568 max_used_connections=39 max_connections=200 threads_connected=11 It is possible that mysqld could use up to key_buffer_size + (record_buffer + sort_buffer)*max_connections = 474330 K bytes of memory Hope that's ok, if not, decrease some variables in the equation Attempting
Re: InnoDB bug?
Mailing-List: contact [EMAIL PROTECTED]; run by ezmlm (http://www.ezmlm.org) From: gbu [EMAIL PROTECTED] I'm experiencing very strange innodb behavior. What version of mysqld? - Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail [EMAIL PROTECTED] To unsubscribe, e-mail [EMAIL PROTECTED] Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
Re: InnoDB bug?
I'm experiencing very strange innodb behavior. What version of mysqld? About test system. FreeBSD 4.2, MySQL 3.23.49, my.cnf innodb settings: === # Uncomment the following if you are using Innobase tables innodb_data_file_path = ibdata1:2M innodb_data_home_dir = /db/mysql/ innodb_log_group_home_dir = /db/mysql/ innodb_log_arch_dir = /db/mysql/ set-variable = innodb_mirrored_log_groups=1 set-variable = innodb_log_files_in_group=3 set-variable = innodb_log_file_size=128M set-variable = innodb_log_buffer_size=64M innodb_flush_log_at_trx_commit=1 innodb_log_archive=0 set-variable = innodb_buffer_pool_size=128M set-variable = innodb_additional_mem_pool_size=16M set-variable = innodb_file_io_threads=4 - Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail [EMAIL PROTECTED] To unsubscribe, e-mail [EMAIL PROTECTED] Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
InnoDB bug?
Hi I'm experiencing very strange innodb behavior. I'm testing the following DB structure: stn|dt|par1|par2|par3 , where stn is 1 to N, dt 1 to M, par1, par2, par3, par4, par5, par6 are any values and stn+dt is a primary key. So create statement looks like CREATE TABLE t3 (stn SMALLINT UNSIGNED NOT NULL, dt BIGINT UNSIGNED NOT NULL, par1 TINYINT, par2 TINYINT, par3 TINYINT, par4 TINYINT, par5 TINYINT, par6 TINYINT, PRIMARY KEY (stn, dt)) TYPE=INNODB The table looks like that: 1 1 9 9 9 9 9 9 1 2 9 9 9 9 9 9 . 2 1 9 9 9 9 9 9 2 2 9 9 9 9 9 9 . N M-1 9 9 9 9 9 9 N M 9 9 9 9 9 9 I'm testing it for N=(4000, 8000) and M=(1500, 3000, 6000, 12000, 24000). Test consits of writing down the file for current N and M, loading it in table via 'LOAD DATA INFILE', commiting changes and then testing speed of 500 random selects from the table. For N=4000 all is loking good, but when N=8000 loading time is droping down drastically when M reaches 3000 or so. I'm talking about 50 times slower insertion here. So I simplified the test, lefting just the loading data in table and run it for N=4000, M=12000 and then for N=8000, M=6000. Notice the data and index length for this cases are the same. The first test was finished in a hour, the second was terminated by me after 12 hour work and it did not even inserted half the rows. If this helps the following are outputs of innodb monitor for this tests. Typical output for N=4000, M=12000: = 020923 3:44:12 INNODB MONITOR OUTPUT = -- SEMAPHORES -- OS WAIT ARRAY INFO: reservation count 1858, signal count 1858 Mutex spin waits 1868, rounds 19710, OS waits 132 RW-shared spins 248, OS waits 124; RW-excl spins 9149, OS waits 1602 TRANSACTIONS Trx id counter 0 1287 Purge done for trx's n:o 0 0 undo n:o 0 0 Total number of lock structs in row lock hash table 0 ---TRANSACTION 0 1286, OS thread id 441071616 inserting, active, runs or sleeps, has 1 lock struct(s), undo log entries 11479793 MySQL thread id 1, query id 16 localhost admin LOAD DATA INFILE '/db/tmp/df.dat' INTO TABLE t3 FIELDS TERMINATED BY ',' LINES TERMINATED BY ' ' FILE I/O I/O thread 0 state: waiting for i/o request I/O thread 1 state: waiting for i/o request I/O thread 2 state: waiting for i/o request I/O thread 3 state: waiting for i/o request Pending normal aio reads: 0, aio writes: 0, ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0 Pending flushes (fsync) log: 0; buffer pool: 0 42975 OS file reads, 18554 OS file writes, 3895 OS fsyncs 0.00 reads/s, 5.88 writes/s, 2.06 fsyncs/s - INSERT BUFFER - Ibuf for space 0: size 1, free list len 0, seg size 2, 0 inserts, 0 merged recs, 0 merges --- LOG --- Log sequence number 0 671108249 Log flushed up to 0 670978039 Last checkpoint at 0 54962 0 pending log writes, 0 pending chkp writes 1360 log i/o's done, 1.56 log i/o's/second -- BUFFER POOL AND MEMORY -- Total memory allocated 246403343; in additional pool allocated 205696 Free list length 0 LRU list length 8187 Flush list length 8001 Buffer pool size 8192 Pending reads 0 Pending writes: LRU 0, flush list 0, single page 0 Pages read 61069, created 55116, written 108019 0.00 reads/s, 23.81 creates/s, 16.56 writes/s Buffer pool hit rate 1000 / 1000 -- ROW OPERATIONS -- 1 queries inside InnoDB, 0 queries in queue; main thread: sleeping Number of rows inserted 11879792, updated 0, deleted 0, read 0 17659.81 inserts/s, 0.00 updates/s, 0.00 deletes/s, 0.00 reads/s END OF INNODB MONITOR OUTPUT 17659.81 inserts/s. Pretty good I guess. While for N=8000, M=6000 at the beginning monitor output looks the same, after ~260 rows insert speed begins falling down, and very fast. And the typical output of monitor becomes like this = 020923 5:04:24 INNODB MONITOR OUTPUT = -- SEMAPHORES -- OS WAIT ARRAY INFO: reservation count 1671, signal count 1670 --Thread 441090048 has waited at btr0cur.c line 357 for 0.00 seconds the semaphore: X-lock on RW-latch at 12bcee78 created in file buf0buf.c line 348 a writer (thread id 441090048) has reserved it in mode exclusive number of readers 0, waiters flag 1 Last time read locked in file buf0flu.c line 466 Last time write locked in file buf0buf.c line 1246 Mutex spin waits 416, rounds 4470, OS waits 42 RW-shared spins 60, OS waits 30; RW-excl spins 4163, OS waits 1599 TRANSACTIONS
Mysql/Innodb bug
Appears to be a database memory corruption problem. The binlog filename had been trashed. Eventually, mysql/innodb blew up trying to do a malloc. We were accidentally running two copies of ibbackup at the same time at around 6pm on 9/21 and 9/22. We were accidentally running a third copy at around 11am on 9/21 and 9/22. The 11am backups appear to have succeeded. One of the 6pm backups reported success on each of 9/21 and 9/22. The binlog index file recorded 2 trashed binlog filenames. A binlog file was created on disk with a trashed filename. The trashed binlog filename has a time-last-modified of 5:05. At first glance it would seem that the fact we were running multiple copies of the backup software was unreleated to the core dump. Stack trace looks like: 0x806eea4 0x82d9218 0x83016a7 0x8301261 0x82c0b8e 0x806bf9d 0x806bc0e 0x808748c 0x809d845 0x8076eee 0x8079e3c 0x8074f14 0x80742c7 0x806eea4 handle_segfault__Fi + 428 0x82d9218 pthread_sighandler + 184 0x83016a7 chunk_alloc + 759 0x8301261 malloc + 209 0x82c0b8e my_malloc + 30 0x806bf9d get_lock_data__FP3THDPP8st_tableUibT1 + 113 0x806bc0e mysql_lock_tables__FP3THDPP8st_tableUi + 574 0x808748c open_ltable__FP3THDP13st_table_list13thr_lock_type + 280 0x809d845 mysql_update__FP3THDP13st_table_listRt4List1Z4ItemT2P4ItemUl15enum_duplicates13thr_lock_type + 53 0x8076eee mysql_execute_command__Fv + 5322 0x8079e3c mysql_parse__FP3THDPcUi + 216 0x8074f14 do_command__FP3THD + 1460 0x80742c7 handle_one_connection__FPv + 655 .err file looks like 020920 10:26:33 mysqld ended 020920 10:32:20 mysqld started 020920 10:32:26 InnoDB: Log file /dblog/email/ib_logfile0 did not exist: new to be created InnoDB: Setting log file /dblog/email/ib_logfile0 size to 48 MB InnoDB: Database physically writes the file full: wait... 020920 10:32:30 InnoDB: Log file /dblog/email/ib_logfile1 did not exist: new to be created InnoDB: Setting log file /dblog/email/ib_logfile1 size to 48 MB InnoDB: Database physically writes the file full: wait... 020920 10:32:36 InnoDB: Log file /dblog/email/ib_logfile2 did not exist: new to be created InnoDB: Setting log file /dblog/email/ib_logfile2 size to 48 MB InnoDB: Database physically writes the file full: wait... 020920 10:32:42 InnoDB: Database was not shut down normally. InnoDB: Starting recovery from log files... InnoDB: Starting log scan based on checkpoint at InnoDB: log sequence number 0 3061070860 InnoDB: Doing recovery: scanned up to log sequence number 0 3061070860 InnoDB: Last MySQL binlog file position 0 5163, file name /data/email/mysql_binlog/binlog.927 020920 10:32:44 InnoDB: Flushing modified pages from the buffer pool... 020920 10:32:44 InnoDB: Started /usr/sbin/mysqld-max: ready for connections mysqld got signal 11; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked agaist is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail key_buffer_size=67104768 record_buffer=1044480 sort_buffer=1048568 max_used_connections=39 max_connections=200 threads_connected=11 It is possible that mysqld could use up to key_buffer_size + (record_buffer + sort_buffer)*max_connections = 474330 K bytes of memory Hope that's ok, if not, decrease some variables in the equation Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... Stack range sanity check OK, backtrace follows: 0x806eea4 0x82d9218 0x83016a7 0x8301261 0x82c0b8e 0x806bf9d 0x806bc0e 0x808748c 0x809d845 0x8076eee 0x8079e3c 0x8074f14 0x80742c7 Stack trace seems successful - bottom reached Please read http://www.mysql.com/doc/U/s/Using_stack_trace.html and follow instructions on how to resolve the stack trace. Resolved stack trace is much more helpful in diagnosing the problem, so please do resolve it Trying to get some variables. Some pointers may be invalid and cause the dump to abort... thd-query at 0x881b288 = UPDATE plx_email set modified = now(), edit_counter = edit_counter+1, is_global = 1 where address_lcase = [EMAIL
Re: innodb bug
Heikki,
Thanks for the patch. As I'm going on vacation tomorrow, I'll give it a
try next week and let you know if I find anything.
--shak
Heikki Tuuri wrote:
Hi!
It turned out that the bug indeed was connected with a 32-bit signed integer
wrap-over if the buffer pool on a 32-bit computer is bigger than 2 GB.
The following patch may fix the problem, but better test first if there are
also other similar wrap-overs which I did not notice.
Best regards,
Heikki
Innobase Oy
--- 1.4/innobase/include/buf0buf.ic Tue Dec 4 16:14:52 2001
+++ 1.5/innobase/include/buf0buf.ic Wed Jun 26 21:42:32 2002
@@ -209,7 +209,7 @@
ut_ad((ulint)ptr = (ulint)frame_zero);
- block = buf_pool_get_nth_block(buf_pool, (ptr - frame_zero)
+ block = buf_pool_get_nth_block(buf_pool, ((ulint)(ptr - frame_zero))
UNIV_PAGE_SIZE_SHIFT);
ut_a(block = buf_pool-blocks);
ut_a(block buf_pool-blocks + buf_pool-max_size);
@@ -236,7 +236,7 @@
ut_ad((ulint)ptr = (ulint)frame_zero);
- block = buf_pool_get_nth_block(buf_pool, (ptr - frame_zero)
+ block = buf_pool_get_nth_block(buf_pool, ((ulint)(ptr - frame_zero))
UNIV_PAGE_SIZE_SHIFT);
ut_a(block = buf_pool-blocks);
ut_a(block buf_pool-blocks + buf_pool-max_size);
- Original Message -
From: Heikki Tuuri [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Wednesday, June 26, 2002 4:42 PM
Subject: Re: innodb bug
Shakeel,
this may be something with 32-bit unsigned integer / signed integer
arithmetic. I assume mysqld runs in the 32-bit mode?
Are you able to compile mysqld yourself? You could add the following to
line
214 of mysql/innobase/include/buf0buf.ic
...
if (block buf_pool-blocks) {
printf(Values %lu, %lu, %lu, %lu\n, (ulint)(ptr - frame_zero),
(ulint)((ptr - frame_zero) UNIV_PAGE_SIZE_SHIFT),
(ulint)block, (ulint)(buf_pool-blocks),
(ulint)ptr, (ulint)frame_zero);
}
...
Regards,
Heikki
Innobase Oy
- Original Message -
From: Shakeel Sorathia [EMAIL PROTECTED]
Newsgroups: mailing.database.mysql
Sent: Wednesday, June 26, 2002 1:19 AM
Subject: innodb bug
I've been having a problem with innodb lately. We just upgraded one of
our machine to have 4 GB of ram in it. However, whenever I make the
innodb_buffer_pool_size greater then 2048M It crashes with the
following in the error log. It's 3.23.51 running on a Solaris 8
Ultrasparc II machine with 4 GB ram. Is the limit 2gb of ram, or is
there something that I'm doing wrong? Thanks for the help!
--shak
020625 12:57:14 mysqld started
InnoDB: Assertion failure in thread 1 in file ../include/buf0buf.ic line
214
InnoDB: We intentionally generate a memory trap.
InnoDB: Send a detailed bug report to [EMAIL PROTECTED]
mysqld got signal 11;
This could be because you hit a bug. It is also possible that this
binary
or one of the libraries it was linked agaist is corrupt, improperly
built,
or misconfigured. This error can also be caused by malfunctioning
hardware.
We will try our best to scrape up some info that will hopefully help
diagnose
the problem, but since we have already crashed, something is definitely
wrong
and this may fail
key_buffer_size=8388600
record_buffer=131072
sort_buffer=1048568
max_used_connections=0
max_connections=1024
threads_connected=0
It is possible that mysqld could use up to
key_buffer_size + (record_buffer + sort_buffer)*max_connections =
1187831
K
bytes of memory
Hope that's ok, if not, decrease some variables in the equation
020625 12:57:54 mysqld ended
--
Shakeel Sorathia
Systems Administrator
(213) 739-5348
-
Before posting, please check:
http://www.mysql.com/manual.php (the manual)
http://lists.mysql.com/ (the list archive)
To request this thread, e-mail [EMAIL PROTECTED]
To unsubscribe, e-mail
[EMAIL PROTECTED]
Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
-
Before posting, please check:
http://www.mysql.com/manual.php (the manual)
http://lists.mysql.com/ (the list archive)
To request this thread, e-mail [EMAIL PROTECTED]
To unsubscribe, e-mail [EMAIL PROTECTED]
Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
--
Shakeel Sorathia
Systems Administrator
(213) 739-5348
-
Before posting, please check:
http://www.mysql.com/manual.php (the manual)
http://lists.mysql.com/ (the list archive)
To request this thread, e-mail [EMAIL PROTECTED]
To unsubscribe, e-mail [EMAIL PROTECTED]
Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
RE: innodb bug
Hi, I'm sort of glad we're not the only one having this problem. Yesterday we had kind of the same error message on an Solaris 8 machine with 512Mb of ram. Our buffer_pool_size was set to 250Mb, because the other 250Mb is used by the orion-web-server. Today I will perform crash-tests on another machine and try to find out the problem. Fyi, our error log: = /opt/nusphere/mysql-max-3.23.49-sun-solaris2.8-sparc/bin/mysqld: ready for connections mysqld got signal 10; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked agaist is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail key_buffer_size=8388600 record_buffer=131072 sort_buffer=2097144 max_used_connections=16 max_connections=100 threads_connected=3 It is possible that mysqld could use up to key_buffer_size + (record_buffer + sort_buffer)*max_connections = 225791 K bytes of memory Hope that's ok, if not, decrease some variables in the equation 020625 15:39:58 mysqld restarted 020625 15:40:34 InnoDB: Database was not shut down normally. InnoDB: Starting recovery from log files... InnoDB: Starting log scan based on checkpoint at InnoDB: log sequence number 0 272046313 InnoDB: Fatal error: cannot allocate 2310548 bytes of InnoDB: memory with malloc! Total allocated memory InnoDB: by InnoDB 334012166 bytes. Operating system errno: 11 InnoDB: Cannot continue operation! InnoDB: Check if you should increase the swap file or InnoDB: ulimits of your operating system. InnoDB: On FreeBSD check you have compiled the OS with InnoDB: a big enough maximum process size. 020625 15:40:37 mysqld ended -Original Message- From: Shakeel Sorathia [mailto:[EMAIL PROTECTED]] Sent: Tuesday, June 25, 2002 21:01 To: [EMAIL PROTECTED] Subject: innodb bug I've been having a problem with innodb lately. We just upgraded one of our machine to have 4 GB of ram in it. However, whenever I make the innodb_buffer_pool_size greater then 2048M It crashes with the following in the error log. It's 3.23.51 running on a Solaris 8 Ultrasparc II machine with 4 GB ram. Is the limit 2gb of ram, or is there something that I'm doing wrong? Thanks for the help! --shak 020625 12:57:14 mysqld started InnoDB: Assertion failure in thread 1 in file ../include/buf0buf.ic line 214 InnoDB: We intentionally generate a memory trap. InnoDB: Send a detailed bug report to [EMAIL PROTECTED] mysqld got signal 11; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked agaist is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail key_buffer_size=8388600 record_buffer=131072 sort_buffer=1048568 max_used_connections=0 max_connections=1024 threads_connected=0 It is possible that mysqld could use up to key_buffer_size + (record_buffer + sort_buffer)*max_connections = 1187831 K bytes of memory Hope that's ok, if not, decrease some variables in the equation 020625 12:57:54 mysqld ended -- Shakeel Sorathia Systems Administrator (213) 739-5348 - Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail [EMAIL PROTECTED] To unsubscribe, e-mail [EMAIL PROTECTED] Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php - Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail [EMAIL PROTECTED] To unsubscribe, e-mail [EMAIL PROTECTED] Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
Re: innodb bug
Ah, that makes sense. So it potentially could be the simple matter of telling the compiler that the type is unsigned. --shak Chuck Simmons wrote: Bert -- Your problem is not the same as Shakeel's. For you, the database is saying that it couldn't allocate memory. For Shakeel, it is saying that an assert failed. At about line 213, there is a right shift (X Y) that is occuring. The behavior of a right shift is different depending on whether the value being shifted is signed or unsigned. The value is supposed to be unsigned, but the programmers forgot to tell the compiler. This effectively means that mysql cannot allocate more than 2GB of ram. block = buf_pool_get_nth_block(buf_pool, (ptr - frame_zero) UNIV_PAGE_SIZE_SHIFT); ut_a(block = buf_pool-blocks); Chuck Bert VdB wrote: Hi, I'm sort of glad we're not the only one having this problem. Yesterday we had kind of the same error message on an Solaris 8 machine with 512Mb of ram. Our buffer_pool_size was set to 250Mb, because the other 250Mb is used by the orion-web-server. Today I will perform crash-tests on another machine and try to find out the problem. Fyi, our error log: = /opt/nusphere/mysql-max-3.23.49-sun-solaris2.8-sparc/bin/mysqld: ready for connections mysqld got signal 10; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked agaist is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail key_buffer_size=8388600 record_buffer=131072 sort_buffer=2097144 max_used_connections=16 max_connections=100 threads_connected=3 It is possible that mysqld could use up to key_buffer_size + (record_buffer + sort_buffer)*max_connections = 225791 K bytes of memory Hope that's ok, if not, decrease some variables in the equation 020625 15:39:58 mysqld restarted 020625 15:40:34 InnoDB: Database was not shut down normally. InnoDB: Starting recovery from log files... InnoDB: Starting log scan based on checkpoint at InnoDB: log sequence number 0 272046313 InnoDB: Fatal error: cannot allocate 2310548 bytes of InnoDB: memory with malloc! Total allocated memory InnoDB: by InnoDB 334012166 bytes. Operating system errno: 11 InnoDB: Cannot continue operation! InnoDB: Check if you should increase the swap file or InnoDB: ulimits of your operating system. InnoDB: On FreeBSD check you have compiled the OS with InnoDB: a big enough maximum process size. 020625 15:40:37 mysqld ended -Original Message- From: Shakeel Sorathia [mailto:[EMAIL PROTECTED]] Sent: Tuesday, June 25, 2002 21:01 To: [EMAIL PROTECTED] Subject: innodb bug I've been having a problem with innodb lately. We just upgraded one of our machine to have 4 GB of ram in it. However, whenever I make the innodb_buffer_pool_size greater then 2048M It crashes with the following in the error log. It's 3.23.51 running on a Solaris 8 Ultrasparc II machine with 4 GB ram. Is the limit 2gb of ram, or is there something that I'm doing wrong? Thanks for the help! --shak 020625 12:57:14 mysqld started InnoDB: Assertion failure in thread 1 in file ../include/buf0buf.ic line 214 InnoDB: We intentionally generate a memory trap. InnoDB: Send a detailed bug report to [EMAIL PROTECTED] mysqld got signal 11; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked agaist is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail key_buffer_size=8388600 record_buffer=131072 sort_buffer=1048568 max_used_connections=0 max_connections=1024 threads_connected=0 It is possible that mysqld could use up to key_buffer_size + (record_buffer + sort_buffer)*max_connections = 1187831 K bytes of memory Hope that's ok, if not, decrease some variables in the equation 020625 12:57:54 mysqld ended - Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail [EMAIL PROTECTED] To unsubscribe, e-mail [EMAIL PROTECTED] Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
Re: innodb bug
Hi!
It turned out that the bug indeed was connected with a 32-bit signed integer
wrap-over if the buffer pool on a 32-bit computer is bigger than 2 GB.
The following patch may fix the problem, but better test first if there are
also other similar wrap-overs which I did not notice.
Best regards,
Heikki
Innobase Oy
--- 1.4/innobase/include/buf0buf.ic Tue Dec 4 16:14:52 2001
+++ 1.5/innobase/include/buf0buf.ic Wed Jun 26 21:42:32 2002
@@ -209,7 +209,7 @@
ut_ad((ulint)ptr = (ulint)frame_zero);
- block = buf_pool_get_nth_block(buf_pool, (ptr - frame_zero)
+ block = buf_pool_get_nth_block(buf_pool, ((ulint)(ptr - frame_zero))
UNIV_PAGE_SIZE_SHIFT);
ut_a(block = buf_pool-blocks);
ut_a(block buf_pool-blocks + buf_pool-max_size);
@@ -236,7 +236,7 @@
ut_ad((ulint)ptr = (ulint)frame_zero);
- block = buf_pool_get_nth_block(buf_pool, (ptr - frame_zero)
+ block = buf_pool_get_nth_block(buf_pool, ((ulint)(ptr - frame_zero))
UNIV_PAGE_SIZE_SHIFT);
ut_a(block = buf_pool-blocks);
ut_a(block buf_pool-blocks + buf_pool-max_size);
- Original Message -
From: Heikki Tuuri [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Wednesday, June 26, 2002 4:42 PM
Subject: Re: innodb bug
Shakeel,
this may be something with 32-bit unsigned integer / signed integer
arithmetic. I assume mysqld runs in the 32-bit mode?
Are you able to compile mysqld yourself? You could add the following to
line
214 of mysql/innobase/include/buf0buf.ic
...
if (block buf_pool-blocks) {
printf(Values %lu, %lu, %lu, %lu\n, (ulint)(ptr - frame_zero),
(ulint)((ptr - frame_zero) UNIV_PAGE_SIZE_SHIFT),
(ulint)block, (ulint)(buf_pool-blocks),
(ulint)ptr, (ulint)frame_zero);
}
...
Regards,
Heikki
Innobase Oy
- Original Message -
From: Shakeel Sorathia [EMAIL PROTECTED]
Newsgroups: mailing.database.mysql
Sent: Wednesday, June 26, 2002 1:19 AM
Subject: innodb bug
I've been having a problem with innodb lately. We just upgraded one of
our machine to have 4 GB of ram in it. However, whenever I make the
innodb_buffer_pool_size greater then 2048M It crashes with the
following in the error log. It's 3.23.51 running on a Solaris 8
Ultrasparc II machine with 4 GB ram. Is the limit 2gb of ram, or is
there something that I'm doing wrong? Thanks for the help!
--shak
020625 12:57:14 mysqld started
InnoDB: Assertion failure in thread 1 in file ../include/buf0buf.ic line
214
InnoDB: We intentionally generate a memory trap.
InnoDB: Send a detailed bug report to [EMAIL PROTECTED]
mysqld got signal 11;
This could be because you hit a bug. It is also possible that this
binary
or one of the libraries it was linked agaist is corrupt, improperly
built,
or misconfigured. This error can also be caused by malfunctioning
hardware.
We will try our best to scrape up some info that will hopefully help
diagnose
the problem, but since we have already crashed, something is definitely
wrong
and this may fail
key_buffer_size=8388600
record_buffer=131072
sort_buffer=1048568
max_used_connections=0
max_connections=1024
threads_connected=0
It is possible that mysqld could use up to
key_buffer_size + (record_buffer + sort_buffer)*max_connections =
1187831
K
bytes of memory
Hope that's ok, if not, decrease some variables in the equation
020625 12:57:54 mysqld ended
--
Shakeel Sorathia
Systems Administrator
(213) 739-5348
-
Before posting, please check:
http://www.mysql.com/manual.php (the manual)
http://lists.mysql.com/ (the list archive)
To request this thread, e-mail [EMAIL PROTECTED]
To unsubscribe, e-mail
[EMAIL PROTECTED]
Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
-
Before posting, please check:
http://www.mysql.com/manual.php (the manual)
http://lists.mysql.com/ (the list archive)
To request this thread, e-mail [EMAIL PROTECTED]
To unsubscribe, e-mail [EMAIL PROTECTED]
Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
Re: innodb bug
Bert -- Your problem is not the same as Shakeel's. For you, the database is saying that it couldn't allocate memory. For Shakeel, it is saying that an assert failed. At about line 213, there is a right shift (X Y) that is occuring. The behavior of a right shift is different depending on whether the value being shifted is signed or unsigned. The value is supposed to be unsigned, but the programmers forgot to tell the compiler. This effectively means that mysql cannot allocate more than 2GB of ram. block = buf_pool_get_nth_block(buf_pool, (ptr - frame_zero) UNIV_PAGE_SIZE_SHIFT); ut_a(block = buf_pool-blocks); Chuck Bert VdB wrote: Hi, I'm sort of glad we're not the only one having this problem. Yesterday we had kind of the same error message on an Solaris 8 machine with 512Mb of ram. Our buffer_pool_size was set to 250Mb, because the other 250Mb is used by the orion-web-server. Today I will perform crash-tests on another machine and try to find out the problem. Fyi, our error log: = /opt/nusphere/mysql-max-3.23.49-sun-solaris2.8-sparc/bin/mysqld: ready for connections mysqld got signal 10; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked agaist is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail key_buffer_size=8388600 record_buffer=131072 sort_buffer=2097144 max_used_connections=16 max_connections=100 threads_connected=3 It is possible that mysqld could use up to key_buffer_size + (record_buffer + sort_buffer)*max_connections = 225791 K bytes of memory Hope that's ok, if not, decrease some variables in the equation 020625 15:39:58 mysqld restarted 020625 15:40:34 InnoDB: Database was not shut down normally. InnoDB: Starting recovery from log files... InnoDB: Starting log scan based on checkpoint at InnoDB: log sequence number 0 272046313 InnoDB: Fatal error: cannot allocate 2310548 bytes of InnoDB: memory with malloc! Total allocated memory InnoDB: by InnoDB 334012166 bytes. Operating system errno: 11 InnoDB: Cannot continue operation! InnoDB: Check if you should increase the swap file or InnoDB: ulimits of your operating system. InnoDB: On FreeBSD check you have compiled the OS with InnoDB: a big enough maximum process size. 020625 15:40:37 mysqld ended -Original Message- From: Shakeel Sorathia [mailto:[EMAIL PROTECTED]] Sent: Tuesday, June 25, 2002 21:01 To: [EMAIL PROTECTED] Subject: innodb bug I've been having a problem with innodb lately. We just upgraded one of our machine to have 4 GB of ram in it. However, whenever I make the innodb_buffer_pool_size greater then 2048M It crashes with the following in the error log. It's 3.23.51 running on a Solaris 8 Ultrasparc II machine with 4 GB ram. Is the limit 2gb of ram, or is there something that I'm doing wrong? Thanks for the help! --shak 020625 12:57:14 mysqld started InnoDB: Assertion failure in thread 1 in file ../include/buf0buf.ic line 214 InnoDB: We intentionally generate a memory trap. InnoDB: Send a detailed bug report to [EMAIL PROTECTED] mysqld got signal 11; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked agaist is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail key_buffer_size=8388600 record_buffer=131072 sort_buffer=1048568 max_used_connections=0 max_connections=1024 threads_connected=0 It is possible that mysqld could use up to key_buffer_size + (record_buffer + sort_buffer)*max_connections = 1187831 K bytes of memory Hope that's ok, if not, decrease some variables in the equation 020625 12:57:54 mysqld ended - Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail [EMAIL PROTECTED] To unsubscribe, e-mail [EMAIL PROTECTED] Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
Re: innodb bug
Shakeel,
this may be something with 32-bit unsigned integer / signed integer
arithmetic. I assume mysqld runs in the 32-bit mode?
Are you able to compile mysqld yourself? You could add the following to line
214 of mysql/innobase/include/buf0buf.ic
...
if (block buf_pool-blocks) {
printf(Values %lu, %lu, %lu, %lu\n, (ulint)(ptr - frame_zero),
(ulint)((ptr - frame_zero) UNIV_PAGE_SIZE_SHIFT),
(ulint)block, (ulint)(buf_pool-blocks),
(ulint)ptr, (ulint)frame_zero);
}
...
Regards,
Heikki
Innobase Oy
- Original Message -
From: Shakeel Sorathia [EMAIL PROTECTED]
Newsgroups: mailing.database.mysql
Sent: Wednesday, June 26, 2002 1:19 AM
Subject: innodb bug
I've been having a problem with innodb lately. We just upgraded one of
our machine to have 4 GB of ram in it. However, whenever I make the
innodb_buffer_pool_size greater then 2048M It crashes with the
following in the error log. It's 3.23.51 running on a Solaris 8
Ultrasparc II machine with 4 GB ram. Is the limit 2gb of ram, or is
there something that I'm doing wrong? Thanks for the help!
--shak
020625 12:57:14 mysqld started
InnoDB: Assertion failure in thread 1 in file ../include/buf0buf.ic line
214
InnoDB: We intentionally generate a memory trap.
InnoDB: Send a detailed bug report to [EMAIL PROTECTED]
mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked agaist is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning
hardware.
We will try our best to scrape up some info that will hopefully help
diagnose
the problem, but since we have already crashed, something is definitely
wrong
and this may fail
key_buffer_size=8388600
record_buffer=131072
sort_buffer=1048568
max_used_connections=0
max_connections=1024
threads_connected=0
It is possible that mysqld could use up to
key_buffer_size + (record_buffer + sort_buffer)*max_connections = 1187831
K
bytes of memory
Hope that's ok, if not, decrease some variables in the equation
020625 12:57:54 mysqld ended
--
Shakeel Sorathia
Systems Administrator
(213) 739-5348
-
Before posting, please check:
http://www.mysql.com/manual.php (the manual)
http://lists.mysql.com/ (the list archive)
To request this thread, e-mail [EMAIL PROTECTED]
To unsubscribe, e-mail
[EMAIL PROTECTED]
Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
-
Before posting, please check:
http://www.mysql.com/manual.php (the manual)
http://lists.mysql.com/ (the list archive)
To request this thread, e-mail [EMAIL PROTECTED]
To unsubscribe, e-mail [EMAIL PROTECTED]
Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
Re:DBI bug, InnoDB bug, MySQL bug, or I'm just plain missing something.
Eric,
it was not immediately clear to me what your application
does. Does one connection update the table and
another connection do the SELECT? Then the problem
might be the consistent read. I have copied below a segment
from the InnoDB manual at www.innodb.com :
When you issue a consistent read, that is, an ordinary SELECT statement,
InnoDB will give your transaction a timepoint according to which your query
sees the database. Thus, if transaction B deletes a row and commits after
your timepoint was assigned, then you will not see the row deleted.
Similarly with inserts and updates.
You can advance your timepoint by committing your transaction and then doing
another SELECT.
This is called multiversioned concurrency control.
User A User B
set autocommit=0; set autocommit=0;
time
| SELECT * FROM t;
| empty set
|INSERT INTO t VALUES (1, 2);
|
v SELECT * FROM t;
empty set
COMMIT;
SELECT * FROM t;
empty set;
COMMIT;
SELECT * FROM t;
--
| 1|2 |
--
Thus user A sees the row inserted by B only when B has committed the insert,
and A has committed his own transaction so that the timepoint is advanced
past the the commit of B.
If you want to see the 'freshest' state of the database, you should use a
locking read:
SELECT * FROM t LOCK IN SHARE MODE;
The problem you might have is that ::dbh-commit() really does not do
a commit. Have you tested it? If you do a ::dbh-commit() after inserting
some rows with autocommit switched off, and then do ::dbh-do(ROLLBACK),
do the rows get inserted or not?
Eric, executing a commit through ::do-(COMMIT) is a perfectly
valid way to do a commit.
By the way, Sinisa, who is responsible for the DBI driver for MySQL?
It might be outdated.
Regards,
Heikki
Copied message:
I've got a part of a complex system that isn't behaving as expected. I've
got one perl program running with autocommit off that creates entries in a
table and commits the changes. At some point, the rows have their state
field changed from 'active' to 'closed', and these changes are
thencommitted.
The next step in the process is the part I'm having problems with. The
next step is supposed to pull up a list of the records that have a state
of 'closed' and process them. It works fine the first time through, or if
I limit the number of results in the select that gathers the list that I
use to process them it will loop through as many times as necessary to
process them all, but once it has processed everything that was available
when the program was first run, it stops and won't see any records that
were updated after the first run.
The interesting part is, if I run the program with autocommit on, it
works fine. Unfortunately, transactions are essential. I found that the
program works if I turn autocommit on but wrap the critical table updates
in $dbh-do(BEGIN;) and $dbh-do(COMMIT;), but I haven't tested it to
make sure there are no side-effects, and I'd rather use the mechanism
it seems that I'm supposed to use with DBI.
Has anyone seen this problem?
Is do'ing the BEGIN and COMMIT valid?
Anyway, here's the table and the second-stage program (trimmed down as far
as possible).
I'm using MySQL 3.23.41, DBI-1.19, andMsql-Mysql-modules-1.2216.
CREATE TABLE LCCer(switchId CHAR(10) BINARY NOT NULL PRIMARY KEY,
activity INT UNSIGNED NOT NULL,dnis CHAR(10),
state ENUM('active','closed','broken','summed','completed',
'archived') NOT NULL,flags
SET('needs_transreq'),
data TEXT NOT NULL,INDEX state (state),
INDEX activity (activity,dnis)) TYPE = INNODB;#!/usr/bin/perluse DBI;
use strict;
my($livedsn)=DBI:mysql:live;mysql_read_default_file=/home/archuser/.my.cnf
;
my($testdsn)=DBI:mysql:test;mysql_read_default_file=/home/archuser/.my.cnf
;
$main::livedbh=DBI-connect($livedsn,undef,undef, { RaiseError = 1,
AutoCommit = 0 });
$main::testdbh=DBI-connect($testdsn,undef,undef, { RaiseError = 1,
AutoCommit = 0 });
$main::dbh=$main::testdbh;while(1){my($rows,$switchId,$dnis,$data,$count
);
my($cmd)=SELECT switchId,dnis,data FROM LCCer WHERE state='closed'
ORDER BY activity;;
my($sth)=$main::dbh-prepare($cmd);$sth-execute();
while(($switchId,$dnis,$data)=$sth-fetchrow_array()){
$main::dbh-do(UPDATE LCCer SET state='summed' where
switchId='$switchId';);
$main::dbh-commit(); $count++;}$sth-finish();if($count1)
{
print(sleeping for 10\n); sleep(10);}}
-
Before posting, please check:
http://www.mysql.com/manual.php (the manual)
http://lists.mysql.com/ (the list archive)
Re:DBI bug, InnoDB bug, MySQL bug, or I'm just plain missing something.
On Fri, 17 Aug 2001, Heikki Tuuri wrote: Problem resolved, thank you very much. It was indeed my missing a concept, description below: it was not immediately clear to me what your application does. Does one connection update the table and another connection do the SELECT? Yes, as it's two seperate programs. We've got a directory of dynamically loadable modules for the second part of the functionality, and the first part of the functionality takes too long to start up, so we've broken the two steps into seperate programs, so that the second part can exit and reload any time any of the loadable modules change, or a new one is added. You can advance your timepoint by committing your transaction and then doing another SELECT. This is what was throwing me. As soon as I did a ::dbh-commit() regardless of whether I updated any records, the SELECT timepoint was getting properly updated. This is the concept I was missing. I've worked with DBs that supported transactions before, but not multiversioning. If you want to see the 'freshest' state of the database, you should use a locking read: SELECT * FROM t LOCK IN SHARE MODE; The problem you might have is that ::dbh-commit() really does not do a commit. Have you tested it? If you do a ::dbh-commit() after inserting some rows with autocommit switched off, and then do ::dbh-do(ROLLBACK), do the rows get inserted or not? It's a real commit, ::dbh-do(ROLLBACK) didn't undo the ::dbh-commit(), so the problem was almost definitely my lack of understanding of the multiversioning of InnoDB. Eric, executing a commit through ::do-(COMMIT) is a perfectly valid way to do a commit. Good to know. I wasn't expecting the autocommit mode to affect SELECTs, so I was thrown off track when ::dbh-do(COMMIT) seemed to have different side-effects than ::dbh-commit(). Nice to know that they are equivelent, it was the AutoCommit on/off that made the difference. - Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail [EMAIL PROTECTED] To unsubscribe, e-mail [EMAIL PROTECTED] Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
innodb bug
Downloaded latest version of MySQL for Win95 (3.23.38-win) on May 21st. After mysqld command this is the response: C:\cd mysql|cd bin|mysqld Innobase: Assertion failure in thread 4293273545 in file M:\mysql-3.23\innobase\ os\os0file.c line 187 Innobase: we intentionally generate a memory trap. Innobase: Send a bug report to [EMAIL PROTECTED] 010523 15:51:48 C:\MYSQL\BIN\MYSQLD.EXE: Got signal 11. Aborting! 010523 15:51:48 Aborting InnoDB: Warning: shutting down not properly started database 010523 15:51:51 C:\MYSQL\BIN\MYSQLD.EXE: Shutdown Complete Can someone help me ? Thanks Claudio
