Re: InnoDB and long semaphore waits
Heikki Tuuri wrote: Not sure you want that, the file is 44MB uncompressed, and only talks about the errors reading communication packets. Makes for some really boring reading ;) The InnoDB error I managed to figure out - I once upped max_connections without doing the math, and the machine was exhausting physical RAM. So I backed off max_connections and the really ugly errors are now gone :) ok, good. It was then a hang caused by excessive OS swapping or something like that. Definitely a 'user error' and not caused by InnoDB or MySQL. But I still do not have a solution to my error logs filling up with the connection error, which is now unfortunately every 1 to 2 seconds... The machines are connected with gigabit ethernet, and have been running without trouble for at least three months - and now are filling up the error logs. Any ideas, anybody? Bueller? I think those ugly warnings will again be removed in the next 4.0 release. They were instroduced to 4.0.xx accidentally. Thanks Heikki, as long as I know it is a known error I am happy to wait (and truncate my error logs)... :^) -- Mitch -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: InnoDB and long semaphore waits
Mitch, - Alkuperäinen viesti - Lähettäjä: "Mitch Pirtle" <[EMAIL PROTECTED]> Vastaanottaja: "Heikki Tuuri" <[EMAIL PROTECTED]> Kopio: <[EMAIL PROTECTED]> Lähetetty: Monday, July 05, 2004 4:26 PM Aihe: Re: InnoDB and long semaphore waits > Heikki Tuuri wrote: > > >Mitch, > > > >please send the FULL .err log to me. > > > > > > Hey Heikki, > > Not sure you want that, the file is 44MB uncompressed, and only talks > about the errors reading communication packets. Makes for some really > boring reading ;) The InnoDB error I managed to figure out - I once > upped max_connections without doing the math, and the machine was > exhausting physical RAM. So I backed off max_connections and the really > ugly errors are now gone :) ok, good. It was then a hang caused by excessive OS swapping or something like that. > But I still do not have a solution to my error logs filling up with the > connection error, which is now unfortunately every 1 to 2 seconds... > The machines are connected with gigabit ethernet, and have been running > without trouble for at least three months - and now are filling up the > error logs. Any ideas, anybody? Bueller? I think those ugly warnings will again be removed in the next 4.0 release. They were instroduced to 4.0.xx accidentally. > -- Mitch Regards, Heikki > >Best regards, > > > >Heikki > > > >- Original Message ----- > >From: "Mitch Pirtle" <[EMAIL PROTECTED]> > >Newsgroups: mailing.database.myodbc > >Sent: Saturday, July 03, 2004 4:41 PM > >Subject: InnoDB and long semaphore waits > > > > > > > > > >>Hi listers, > >> > >>I just got here, so please let me know if this is not the appropriate > >>list! :) > >> > >>Running MySQL 4.0.20 on Fedora Core 1, with InnoDB tables. Installed > >>from RPMs provided at MySQL.com. > >> > >>Last night the beastie came down hard, and requred a physical reboot in > >>order to free/kill some mysqld processes. I say this as I worry that it > >>could be from a hardware error, however it is running on a dual Xeon > >>machine that is not even 6 months old... > >> > >>When I attempted to get the stack trace I only get "nm: > >>/usr/sbin/mysqld: no symbols", and the docs point at stuff that is not > >>on this box. ? > >> > >>I see two errors that need resolution: > >> > >>-- > >> > >> > >--- > > > > > >>1) In the error log, I see the following: > >> > >>InnoDB: ## Diagnostic info printed to the standard error stream > >>InnoDB: Warning: a long semaphore wait: > >>--Thread 23207961 has waited at btr0sea.c line 480 for 625.00 seconds > >>the semaphore: > >>X-lock on RW-latch at 0x4506a768 created in file btr0sea.c line 139 > >>a writer (thread id 23207961) has reserved it in mode wait exclusive > >>number of readers 1, waiters flag 1 > >>Last time read locked in file btr0sea.c line 745 > >>Last time write locked in file btr0sea.c line 480 > >>InnoDB: Error: semaphore wait has lasted > 600 seconds > >>InnoDB: We intentionally crash the server, because it appears to be hung. > >>040702 20:45:27InnoDB: Assertion failure in thread 24583 in file > >>sync0arr.c line 925 > >>InnoDB: We intentionally generate a memory trap. > >>InnoDB: Submit a detailed bug report to http://bugs.mysql.com. > >>InnoDB: If you get repeated assertion failures or crashes, even > >>InnoDB: immediately after the mysqld startup, there may be > >>InnoDB: corruption in the InnoDB tablespace. See section 6.1 of > >>InnoDB: http://www.innodb.com/ibman.php about forcing recovery. > >>mysqld got signal 11; > >> > >>...and goes on to say: > >> > >>key_buffer_size=402653184 > >>read_buffer_size=2093056 > >>max_used_connections=176 > >>max_connections=500 > >>threads_connected=146 > >>It is possible that mysqld could use up to > >>key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections > >>= 2439212 K > >>bytes of memory > >>Hope that's ok; if not, decrease some variables in the equation. > >> > >>thd=(nil) > >>Attempting backtrace. You can use the following information to find out > >>where mysqld died. If you see no messages after this, something went > >>terrib
Re: InnoDB and long semaphore waits
Heikki Tuuri wrote: Mitch, please send the FULL .err log to me. Hey Heikki, Not sure you want that, the file is 44MB uncompressed, and only talks about the errors reading communication packets. Makes for some really boring reading ;) The InnoDB error I managed to figure out - I once upped max_connections without doing the math, and the machine was exhausting physical RAM. So I backed off max_connections and the really ugly errors are now gone :) But I still do not have a solution to my error logs filling up with the connection error, which is now unfortunately every 1 to 2 seconds... The machines are connected with gigabit ethernet, and have been running without trouble for at least three months - and now are filling up the error logs. Any ideas, anybody? Bueller? -- Mitch Best regards, Heikki - Original Message - From: "Mitch Pirtle" <[EMAIL PROTECTED]> Newsgroups: mailing.database.myodbc Sent: Saturday, July 03, 2004 4:41 PM Subject: InnoDB and long semaphore waits Hi listers, I just got here, so please let me know if this is not the appropriate list! :) Running MySQL 4.0.20 on Fedora Core 1, with InnoDB tables. Installed from RPMs provided at MySQL.com. Last night the beastie came down hard, and requred a physical reboot in order to free/kill some mysqld processes. I say this as I worry that it could be from a hardware error, however it is running on a dual Xeon machine that is not even 6 months old... When I attempted to get the stack trace I only get "nm: /usr/sbin/mysqld: no symbols", and the docs point at stuff that is not on this box. ? I see two errors that need resolution: -- --- 1) In the error log, I see the following: InnoDB: ## Diagnostic info printed to the standard error stream InnoDB: Warning: a long semaphore wait: --Thread 23207961 has waited at btr0sea.c line 480 for 625.00 seconds the semaphore: X-lock on RW-latch at 0x4506a768 created in file btr0sea.c line 139 a writer (thread id 23207961) has reserved it in mode wait exclusive number of readers 1, waiters flag 1 Last time read locked in file btr0sea.c line 745 Last time write locked in file btr0sea.c line 480 InnoDB: Error: semaphore wait has lasted > 600 seconds InnoDB: We intentionally crash the server, because it appears to be hung. 040702 20:45:27InnoDB: Assertion failure in thread 24583 in file sync0arr.c line 925 InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed bug report to http://bugs.mysql.com. InnoDB: If you get repeated assertion failures or crashes, even InnoDB: immediately after the mysqld startup, there may be InnoDB: corruption in the InnoDB tablespace. See section 6.1 of InnoDB: http://www.innodb.com/ibman.php about forcing recovery. mysqld got signal 11; ...and goes on to say: key_buffer_size=402653184 read_buffer_size=2093056 max_used_connections=176 max_connections=500 threads_connected=146 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 2439212 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. thd=(nil) Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... Cannot determine thread, fp=0xbfedf758, backtrace may not be correct. Stack range sanity check OK, backtrace follows: 0x80720d4 0x8250d48 0x81ed044 0x80f9148 0x824e4fc 0x828452a New value of fp=(nil) failed sanity check, terminating stack trace! Again, I try to follow the instructons on dealing with the stack trace but the instructions fail, and also do not provide enough explanation for me to figure out on my own how to fix it :( -- --- 2) After rebooting the server, the error log fills up with: 040703 9:07:59 Aborted connection 55 to db: 'db1' user: 'www' host: `192.168.1.1' (Got an error reading communication packets) ...and it repeats itself roughly every 5-to-10 seconds, which I obviously find alarming. Cannot find any reference on that error, and don't really have enough data to know how to react... Can somebody please whack me upside the head with a cluestick? -- Mitch -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe: http://lists.mysql.com/[EMAIL PROTECTED] -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: InnoDB and long semaphore waits
Mitch, please send the FULL .err log to me. Best regards, Heikki - Original Message - From: "Mitch Pirtle" <[EMAIL PROTECTED]> Newsgroups: mailing.database.myodbc Sent: Saturday, July 03, 2004 4:41 PM Subject: InnoDB and long semaphore waits > Hi listers, > > I just got here, so please let me know if this is not the appropriate > list! :) > > Running MySQL 4.0.20 on Fedora Core 1, with InnoDB tables. Installed > from RPMs provided at MySQL.com. > > Last night the beastie came down hard, and requred a physical reboot in > order to free/kill some mysqld processes. I say this as I worry that it > could be from a hardware error, however it is running on a dual Xeon > machine that is not even 6 months old... > > When I attempted to get the stack trace I only get "nm: > /usr/sbin/mysqld: no symbols", and the docs point at stuff that is not > on this box. ? > > I see two errors that need resolution: > > -- --- > > 1) In the error log, I see the following: > > InnoDB: ## Diagnostic info printed to the standard error stream > InnoDB: Warning: a long semaphore wait: > --Thread 23207961 has waited at btr0sea.c line 480 for 625.00 seconds > the semaphore: > X-lock on RW-latch at 0x4506a768 created in file btr0sea.c line 139 > a writer (thread id 23207961) has reserved it in mode wait exclusive > number of readers 1, waiters flag 1 > Last time read locked in file btr0sea.c line 745 > Last time write locked in file btr0sea.c line 480 > InnoDB: Error: semaphore wait has lasted > 600 seconds > InnoDB: We intentionally crash the server, because it appears to be hung. > 040702 20:45:27InnoDB: Assertion failure in thread 24583 in file > sync0arr.c line 925 > InnoDB: We intentionally generate a memory trap. > InnoDB: Submit a detailed bug report to http://bugs.mysql.com. > InnoDB: If you get repeated assertion failures or crashes, even > InnoDB: immediately after the mysqld startup, there may be > InnoDB: corruption in the InnoDB tablespace. See section 6.1 of > InnoDB: http://www.innodb.com/ibman.php about forcing recovery. > mysqld got signal 11; > > ...and goes on to say: > > key_buffer_size=402653184 > read_buffer_size=2093056 > max_used_connections=176 > max_connections=500 > threads_connected=146 > It is possible that mysqld could use up to > key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections > = 2439212 K > bytes of memory > Hope that's ok; if not, decrease some variables in the equation. > > thd=(nil) > Attempting backtrace. You can use the following information to find out > where mysqld died. If you see no messages after this, something went > terribly wrong... > Cannot determine thread, fp=0xbfedf758, backtrace may not be correct. > Stack range sanity check OK, backtrace follows: > 0x80720d4 > 0x8250d48 > 0x81ed044 > 0x80f9148 > 0x824e4fc > 0x828452a > New value of fp=(nil) failed sanity check, terminating stack trace! > > Again, I try to follow the instructons on dealing with the stack trace > but the instructions fail, and also do not provide enough explanation > for me to figure out on my own how to fix it :( > > -- --- > 2) After rebooting the server, the error log fills up with: > > 040703 9:07:59 Aborted connection 55 to db: 'db1' user: 'www' host: > `192.168.1.1' (Got an error reading communication packets) > ...and it repeats itself roughly every 5-to-10 seconds, which I > obviously find alarming. > > Cannot find any reference on that error, and don't really have enough > data to know how to react... > > > Can somebody please whack me upside the head with a cluestick? > > -- Mitch > > -- > MySQL General Mailing List > For list archives: http://lists.mysql.com/mysql > To unsubscribe: http://lists.mysql.com/[EMAIL PROTECTED] > -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
InnoDB and long semaphore waits
Hi listers, I just got here, so please let me know if this is not the appropriate list! :) Running MySQL 4.0.20 on Fedora Core 1, with InnoDB tables. Installed from RPMs provided at MySQL.com. Last night the beastie came down hard, and requred a physical reboot in order to free/kill some mysqld processes. I say this as I worry that it could be from a hardware error, however it is running on a dual Xeon machine that is not even 6 months old... When I attempted to get the stack trace I only get "nm: /usr/sbin/mysqld: no symbols", and the docs point at stuff that is not on this box. ? I see two errors that need resolution: - 1) In the error log, I see the following: InnoDB: ## Diagnostic info printed to the standard error stream InnoDB: Warning: a long semaphore wait: --Thread 23207961 has waited at btr0sea.c line 480 for 625.00 seconds the semaphore: X-lock on RW-latch at 0x4506a768 created in file btr0sea.c line 139 a writer (thread id 23207961) has reserved it in mode wait exclusive number of readers 1, waiters flag 1 Last time read locked in file btr0sea.c line 745 Last time write locked in file btr0sea.c line 480 InnoDB: Error: semaphore wait has lasted > 600 seconds InnoDB: We intentionally crash the server, because it appears to be hung. 040702 20:45:27InnoDB: Assertion failure in thread 24583 in file sync0arr.c line 925 InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed bug report to http://bugs.mysql.com. InnoDB: If you get repeated assertion failures or crashes, even InnoDB: immediately after the mysqld startup, there may be InnoDB: corruption in the InnoDB tablespace. See section 6.1 of InnoDB: http://www.innodb.com/ibman.php about forcing recovery. mysqld got signal 11; ...and goes on to say: key_buffer_size=402653184 read_buffer_size=2093056 max_used_connections=176 max_connections=500 threads_connected=146 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 2439212 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. thd=(nil) Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... Cannot determine thread, fp=0xbfedf758, backtrace may not be correct. Stack range sanity check OK, backtrace follows: 0x80720d4 0x8250d48 0x81ed044 0x80f9148 0x824e4fc 0x828452a New value of fp=(nil) failed sanity check, terminating stack trace! Again, I try to follow the instructons on dealing with the stack trace but the instructions fail, and also do not provide enough explanation for me to figure out on my own how to fix it :( - 2) After rebooting the server, the error log fills up with: 040703 9:07:59 Aborted connection 55 to db: 'db1' user: 'www' host: `192.168.1.1' (Got an error reading communication packets) ...and it repeats itself roughly every 5-to-10 seconds, which I obviously find alarming. Cannot find any reference on that error, and don't really have enough data to know how to react... Can somebody please whack me upside the head with a cluestick? -- Mitch -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
