I have a system that has been in production for more than a year and fairly reliable.
I have a Penguin Systems Relion 120 running Redhat Linux 2.2.14 and MySQL 3.23.48 A few weeks ago it started crashing occasionally. About a week ago it started crashing a lot. Some times it gets a core dump with a stack back trace: [mwexler@spode ~]$ more /tmp/trace.*.res :::::::::::::: /tmp/trace.10.res :::::::::::::: 0x806edf4 handle_segfault__Fi + 428 0x811fd28 pthread_sighandler + 184 0x810cdc2 hash_next + 62 0x8086aa7 wait_for_tables__FP3THD + 271 0x806bb4f mysql_lock_tables__FP3THDPP8st_tableUi + 543 0x80870b4 lock_tables__FP3THDP13st_table_list + 108 0x808950d open_and_lock_tables__FP3THDP13st_table_list + 41 0x8075982 mysql_execute_command__Fv + 702 0x8079a8c mysql_parse__FP3THDPcUi + 216 0x8074bb4 do_command__FP3THD + 1460 0x8073f67 handle_one_connection__FPv + 655 :::::::::::::: /tmp/trace.11.res :::::::::::::: 0x806edf4 handle_segfault__Fi + 428 0x811fd28 pthread_sighandler + 184 0x811c412 pthread_cond_wait + 226 0x8085f9c open_table__FP3THDPCcN21Pb + 648 0x8086f46 open_ltable__FP3THDP13st_table_list13thr_lock_type + 66 0x809a133 mysql_insert__FP3THDP13st_table_listRt4List1Z4ItemRt4List1Zt4List1Z4It em15enum_duplicates13thr_lock_type + 315 0x8076bea mysql_execute_command__Fv + 5414 0x8079a8c mysql_parse__FP3THDPcUi + 216 0x8074bb4 do_command__FP3THD + 1460 0x8073f67 handle_one_connection__FPv + 655 :::::::::::::: /tmp/trace.12.res :::::::::::::: 0x806edf4 handle_segfault__Fi + 428 0x811fd28 pthread_sighandler + 184 0x810cdc2 hash_next + 62 0x8086aa7 wait_for_tables__FP3THD + 271 0x806bb4f mysql_lock_tables__FP3THDPP8st_tableUi + 543 0x80870b4 lock_tables__FP3THDP13st_table_list + 108 0x808950d open_and_lock_tables__FP3THDP13st_table_list + 41 0x8075f42 mysql_execute_command__Fv + 2174 0x8079a8c mysql_parse__FP3THDPcUi + 216 0x8074bb4 do_command__FP3THD + 1460 0x8073f67 handle_one_connection__FPv + 655 :::::::::::::: /tmp/trace.13.res :::::::::::::: 0x806edf4 handle_segfault__Fi + 428 0x811fd28 pthread_sighandler + 184 0x810cdc2 hash_next + 62 0x8086aa7 wait_for_tables__FP3THD + 271 0x806bb4f mysql_lock_tables__FP3THDPP8st_tableUi + 543 0x80870b4 lock_tables__FP3THDP13st_table_list + 108 0x808950d open_and_lock_tables__FP3THDP13st_table_list + 41 0x8075982 mysql_execute_command__Fv + 702 0x8079a8c mysql_parse__FP3THDPcUi + 216 0x8074bb4 do_command__FP3THD + 1460 0x8073f67 handle_one_connection__FPv + 655 :::::::::::::: /tmp/trace.14.res :::::::::::::: 0x806edf4 handle_segfault__Fi + 428 0x811fd28 pthread_sighandler + 184 0x814860d chunk_alloc + 1917 0x8147d41 malloc + 209 0x810751e my_malloc + 30 0x80871e1 open_temporary_table__FP3THDPCcN21b + 173 0x80bcc02 mysql_create_table__FP3THDPCcT1P24st_ha_create_informationRt4List1Z12c reate_fieldRt4List1Z3KeybT6 + 3870 0x80bcfba create_table_from_items__FP3THDP24st_ha_create_informationPCcT2Pt4List 1Z12create_fieldPt4List1Z3KeyPt4List1Z4ItemPP13st_mysql_l + 510 0x809cc4c prepare__13select_createRt4List1Z4Item + 48 0x808c40c mysql_select__FP3THDP13st_table_listRt4List1Z4ItemP4ItemP8st_orderT4T3 T4UiP13select_result + 1148 0x80760ce mysql_execute_command__Fv + 2570 0x8079a8c mysql_parse__FP3THDPcUi + 216 0x8074bb4 do_command__FP3THD + 1460 0x8073f67 handle_one_connection__FPv + 655 :::::::::::::: /tmp/trace.15.res :::::::::::::: 0x80d51bf MD5Transform + 971 0x40036f75 _end + 937186841 0x4207add2 _end + 971019382 0x4207ac24 _end + 971018952 0x8300d7f _end + 586787 0x8116628 sstep + 392 0x80f4aed underflow + 521 0x80f551c underflow + 3128 0x80f3a8e mi_unique_hash + 198 0x80dc0bd update_dynamic_record + 173 0x80ded65 _nisam_balance_page + 1521 0x80dacef uf_space_prespace + 163 0x80da2a9 _nisam_read_pack_info + 1177 :::::::::::::: /tmp/trace.1.res :::::::::::::: 0x806e214 handle_segfault__Fi + 428 0x811e2b8 pthread_sighandler + 184 (nil) 0x8092021 create_tmp_table__FP3THDP15TMP_TABLE_PARAMRt4List1Z4ItemP8st_orderbN24 Ui + 1649 0x808c1bb mysql_select__FP3THDP13st_table_listRt4List1Z4ItemP4ItemP8st_orderT4T3 T4UiP13select_result + 3579 0x8074d90 mysql_execute_command__Fv + 812 0x8078ecc mysql_parse__FP3THDPcUi + 216 0x8073f54 do_command__FP3THD + 1460 0x8073307 handle_one_connection__FPv + 639 :::::::::::::: /tmp/trace.2.res :::::::::::::: 0x806e214 handle_segfault__Fi + 428 0x811e2b8 pthread_sighandler + 184 0x8091788 create_tmp_field__FP8st_tableP4ItemQ24Item4TypePPP17Item_result_fieldP P5FieldbT5 + 956 0x8092021 create_tmp_table__FP3THDP15TMP_TABLE_PARAMRt4List1Z4ItemP8st_orderbN24 Ui + 1649 0x808c1bb mysql_select__FP3THDP13st_table_listRt4List1Z4ItemP4ItemP8st_orderT4T3 T4UiP13select_result + 3579 0x8074d90 mysql_execute_command__Fv + 812 0x8078ecc mysql_parse__FP3THDPcUi + 216 0x8073f54 do_command__FP3THD + 1460 0x8073307 handle_one_connection__FPv + 639 :::::::::::::: /tmp/trace.3.res :::::::::::::: 0x806e214 handle_segfault__Fi + 428 0x811e2b8 pthread_sighandler + 184 0x8091788 create_tmp_field__FP8st_tableP4ItemQ24Item4TypePPP17Item_result_fieldP P5FieldbT5 + 956 0x8092021 create_tmp_table__FP3THDP15TMP_TABLE_PARAMRt4List1Z4ItemP8st_orderbN24 Ui + 1649 0x808c1bb mysql_select__FP3THDP13st_table_listRt4List1Z4ItemP4ItemP8st_orderT4T3 T4UiP13select_result + 3579 0x8074d90 mysql_execute_command__Fv + 812 0x8078ecc mysql_parse__FP3THDPcUi + 216 0x8073f54 do_command__FP3THD + 1460 0x8073307 handle_one_connection__FPv + 639 :::::::::::::: /tmp/trace.4.res :::::::::::::: 0x806e214 handle_segfault__Fi + 428 0x811e2b8 pthread_sighandler + 184 0x8091766 create_tmp_field__FP8st_tableP4ItemQ24Item4TypePPP17Item_result_fieldP P5FieldbT5 + 922 0x8092021 create_tmp_table__FP3THDP15TMP_TABLE_PARAMRt4List1Z4ItemP8st_orderbN24 Ui + 1649 0x808c1bb mysql_select__FP3THDP13st_table_listRt4List1Z4ItemP4ItemP8st_orderT4T3 T4UiP13select_result + 3579 0x8074d90 mysql_execute_command__Fv + 812 0x8078ecc mysql_parse__FP3THDPcUi + 216 0x8073f54 do_command__FP3THD + 1460 0x8073307 handle_one_connection__FPv + 639 :::::::::::::: /tmp/trace.5.res :::::::::::::: 0x806e214 handle_segfault__Fi + 428 0x811e2b8 pthread_sighandler + 184 0x8634 (?) 0x8092021 create_tmp_table__FP3THDP15TMP_TABLE_PARAMRt4List1Z4ItemP8st_orderbN24 Ui + 1649 0x808c1bb mysql_select__FP3THDP13st_table_listRt4List1Z4ItemP4ItemP8st_orderT4T3 T4UiP13select_result + 3579 0x8074d90 mysql_execute_command__Fv + 812 0x8078ecc mysql_parse__FP3THDPcUi + 216 0x8073f54 do_command__FP3THD + 1460 0x8073307 handle_one_connection__FPv + 639 :::::::::::::: /tmp/trace.6.res :::::::::::::: 0x806edf4 handle_segfault__Fi + 428 0x811fd28 pthread_sighandler + 184 0x80ac556 str_to_TIME__FPCcUiP7st_timeb + 326 0x80acf06 str_to_timestamp__FPCcUi + 30 0x8058eaf val_int__24Item_func_unix_timestamp + 175 0x805032a val_str__13Item_int_funcP6String + 26 0x8049c28 send__4ItemP6String + 76 0x8068afd send_data__11select_sendRt4List1Z4Item + 101 0x8099769 end_send__FP4JOINP13st_join_tableb + 117 0x80943b7 sub_select__FP4JOINP13st_join_tableb + 255 0x80943b7 sub_select__FP4JOINP13st_join_tableb + 255 0x80940c3 do_select__FP4JOINPt4List1Z4ItemP8st_tableP9Procedure + 415 0x808d4db mysql_select__FP3THDP13st_table_listRt4List1Z4ItemP4ItemP8st_orderT4T3 T4UiP13select_result + 5451 0x80759f0 mysql_execute_command__Fv + 812 0x8079a8c mysql_parse__FP3THDPcUi + 216 0x8074bb4 do_command__FP3THD + 1460 0x8073f67 handle_one_connection__FPv + 655 :::::::::::::: /tmp/trace.7.res :::::::::::::: 0x806edf4 handle_segfault__Fi + 428 0x811fd28 pthread_sighandler + 184 0x810c358 hash_search + 72 0x8086a6b wait_for_tables__FP3THD + 211 0x806bb4f mysql_lock_tables__FP3THDPP8st_tableUi + 543 0x80870b4 lock_tables__FP3THDP13st_table_list + 108 0x808950d open_and_lock_tables__FP3THDP13st_table_list + 41 0x8075982 mysql_execute_command__Fv + 702 0x8079a8c mysql_parse__FP3THDPcUi + 216 0x8074bb4 do_command__FP3THD + 1460 0x8073f67 handle_one_connection__FPv + 655 :::::::::::::: /tmp/trace.8.res :::::::::::::: 0x806edf4 handle_segfault__Fi + 428 0x811fd28 pthread_sighandler + 184 0x810cdc2 hash_next + 62 0x8086aa7 wait_for_tables__FP3THD + 271 0x806bb4f mysql_lock_tables__FP3THDPP8st_tableUi + 543 0x80870b4 lock_tables__FP3THDP13st_table_list + 108 0x808950d open_and_lock_tables__FP3THDP13st_table_list + 41 0x8075f42 mysql_execute_command__Fv + 2174 0x8079a8c mysql_parse__FP3THDPcUi + 216 0x8074bb4 do_command__FP3THD + 1460 0x8073f67 handle_one_connection__FPv + 655 :::::::::::::: /tmp/trace.9.res :::::::::::::: 0x806edf4 handle_segfault__Fi + 428 0x811fd28 pthread_sighandler + 184 0x811c412 pthread_cond_wait + 226 0x8085f9c open_table__FP3THDPCcN21Pb + 648 0x8086dd1 open_tables__FP3THDP13st_table_list + 85 0x80894fc open_and_lock_tables__FP3THDP13st_table_list + 24 0x8075982 mysql_execute_command__Fv + 702 0x8079a8c mysql_parse__FP3THDPcUi + 216 0x8074bb4 do_command__FP3THD + 1460 0x8073f67 handle_one_connection__FPv + 655 Sometimes a single process takes over 99% of the CPU and the system stops responding. Even to "show processlist" Sometimes it dies with mysqld process hanging, pid 7117 - killed I have tried shutting down the system and running myisamchk -rq *.MYI on every table. I have tried upgrading to MySQL 3.23.51. I have tried upgrading to RedHat Linux 7.3/Kernel 2.4.18-5 I have tried switching to a different Relion 120 machine running Linux 2.2.19. The first is a system with an AMI Megaraid. The second is an AMI system. Nothing seems to help. I have login support from MySQL AB. And while they have given me lots of suggestions and possibilities to try, none of them have solved the problem yet. Any advice would be greatly appreciated. --------------------------------------------------------------------- Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail <[EMAIL PROTECTED]> To unsubscribe, e-mail <[EMAIL PROTECTED]> Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
