reason for magic crashes.
i've got third crash third week in a row. Every time in sunday after 18:00, every time with rsync process (which means rsync based backup that is done every day, not just in sunday!), you may see a crash (viewed from KVM) at http://www.tensor.gdynia.pl/~wojtek/crash.png what is important - syncing disk doesn't go on, system hangs here. For 99% system is not overheating at sunday, but i will be 100% sure as i added ipmitool sensor logged from cron every 5 minutes. Please give me an idea what to check. There is nothing in cron that is done at sunday. i don't run periodic stuff in /etc/crontab ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: reason for magic crashes.
On Sun, Jun 24, 2012 at 7:05 PM, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl wrote: i've got third crash third week in a row. From you 5 days ago[1]: it is unimportant as FreeBSD don't crash. Man, I really don't understand a thing... [1] http://freebsd.1045724.n5.nabble.com/Replacing-rc-8-Was-FreeBSD-Boot-Times-td5718636.html Every time in sunday after 18:00, every time with rsync process (which means rsync based backup that is done every day, not just in sunday!), you may see a crash (viewed from KVM) at http://www.tensor.gdynia.pl/~wojtek/crash.png what is important - syncing disk doesn't go on, system hangs here. For 99% system is not overheating at sunday, but i will be 100% sure as i added ipmitool sensor logged from cron every 5 minutes. Please give me an idea what to check. There is nothing in cron that is done at sunday. i don't run periodic stuff in /etc/crontab ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: reason for magic crashes.
[1] http://freebsd.1045724.n5.nabble.com/Replacing-rc-8-Was-FreeBSD-Boot-Times-td5718636.html this 2 minute boot time that will follow doesn't matter as it doesn't crash every now and then - it is nothing compared to the fact you have to travel there. Please give me an idea what to check. There is nothing in cron that is done at sunday. i don't run periodic stuff in /etc/crontab any idea to help? ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: reason for magic crashes.
At 19:05 24/06/2012, Wojciech Puchar wrote: i've got third crash third week in a row. Every time in sunday after 18:00, every time with rsync process (which means rsync based backup that is done every day, not just in sunday!), Is it the same rsync everyday, including sundays, or the sunday rsync is different? Perhaps you have some part of the filesystem corrupted or hd damaged zone and the sundays rsync is the only one that backups/touchs that part. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: reason for magic crashes.
On Sun, Jun 24, 2012 at 07:05:35PM +0200, Wojciech Puchar wrote: i've got third crash third week in a row. Every time in sunday after 18:00, every time with rsync process (which means rsync based backup that is done every day, not just in sunday!), you may see a crash (viewed from KVM) at http://www.tensor.gdynia.pl/~wojtek/crash.png what is important - syncing disk doesn't go on, system hangs here. For 99% system is not overheating at sunday, but i will be 100% sure as i added ipmitool sensor logged from cron every 5 minutes. Please give me an idea what to check. There is nothing in cron that is done at sunday. i don't run periodic stuff in /etc/crontab Compile the kernel with the following: makeoptions DEBUG=-O0 -g options KDB # Enable kernel debugger support. options DDB # Support DDB. options GDB # Support remote GDB. options DEADLKRES # Enable the deadlock resolver options INVARIANTS # Enable calls of extra sanity checking options INVARIANT_SUPPORT # Extra sanity checks of internal structures, required by INVARIANTS options WITNESS # Enable checks to detect deadlocks and cycles options WITNESS_SKIPSPIN# Don't run witness on spinlocks for speed options DIAGNOSTIC After kernel panic ddb prompt will be waiting for you. Type in: dump enter reset enter Make sure you have swap that can handle crashdumps. See this for more details: http://www.freebsd.org/doc/en/books/developers-handbook/kerneldebug.html You can check if everything works correctly by issuing panic manually: sysctl debug.kdb.panic=1 then typing aforementioned ddb commands. After reboot you should get core in /var/crash. Also provide the following: - system version - filesystems involved in rsync with mount details (e.g. UFS with SU+J) - dmesg Hopefully this will be enough for someone to help. -- Mateusz Guzik mjguzik gmail.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: reason for magic crashes.
Have you proven beyond reasonable doubt that there is no filesystem corruption or silent filesystem corruption due to bad hardware? ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: reason for magic crashes.
On 24/06/2012 18:05, Wojciech Puchar wrote: i've got third crash third week in a row. Every time in sunday after 18:00, every time with rsync process (which means rsync based backup that is done every day, not just in sunday!), you may see a crash (viewed from KVM) at http://www.tensor.gdynia.pl/~wojtek/crash.png what is important - syncing disk doesn't go on, system hangs here. For 99% system is not overheating at sunday, but i will be 100% sure as i added ipmitool sensor logged from cron every 5 minutes. Please give me an idea what to check. From the FAQ http://www.freebsd.org/doc/en/books/faq/troubleshoot.html#TRAP-12-PANIC and http://www.freebsd.org/doc/en/books/faq/advanced.html#KERNEL-PANIC-TROUBLESHOOTING Hope that helps. Vince There is nothing in cron that is done at sunday. i don't run periodic stuff in /etc/crontab ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: reason for magic crashes.
There is nothing in cron that is done at sunday. i don't run periodic stuff in /etc/crontab Compile the kernel with the following: makeoptions DEBUG=-O0 -g options KDB # Enable kernel debugger support. options DDB # Support DDB. options GDB # Support remote GDB. options DEADLKRES # Enable the deadlock resolver options INVARIANTS # Enable calls of extra sanity checking options INVARIANT_SUPPORT # Extra sanity checks of internal structures, required by INVARIANTS options WITNESS # Enable checks to detect deadlocks and cycles options WITNESS_SKIPSPIN# Don't run witness on spinlocks for speed options DIAGNOSTIC After kernel panic ddb prompt will be waiting for you. Type in: dump enter reset enter Make sure you have swap that can handle crashdumps. already did this part and debug part, but not DDB. As you see - hang not crashdump how much would it slow down whole thing? If less than 2 times it can be - CPU are rerely half loaded ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: reason for magic crashes.
i've got third crash third week in a row. Every time in sunday after 18:00, every time with rsync process (which means rsync based backup that is done every day, not just in sunday!), Is it the same rsync everyday, including sundays, or the sunday rsync is different? the funny part is that it is exactly the same. Perhaps you have some part of the filesystem corrupted or hd damaged zone and the sundays rsync is the only one that backups/touchs that part. full fsck_ffs was done week ago ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: reason for magic crashes.
Have you proven beyond reasonable doubt that there is no filesystem corruption or silent filesystem corruption due to bad hardware? after last crash fsck_ffs found nothing suggesting such a case. Actually the only change i made to this system (running flawless close to two years) is upgrading to latest 8.* from sources less than month ago. but still - even assuming that system update introduced a bug - i cannot understand why it happens at sunday when it is least loaded. Only rarely visited WWW that time, few mails, and no load present as in work days. since last crash i inserted pendrive and set it as dump device (FreeBSD doesn't seem to reliably crashdump to gmirrored+geli devices, i don't have others available). but as you see it halted, no crash dump ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: reason for magic crashes.
On Sun, Jun 24, 2012 at 08:50:41PM +0200, Wojciech Puchar wrote: There is nothing in cron that is done at sunday. i don't run periodic stuff in /etc/crontab Compile the kernel with the following: makeoptions DEBUG=-O0 -g options KDB # Enable kernel debugger support. options DDB # Support DDB. options GDB # Support remote GDB. options DEADLKRES # Enable the deadlock resolver options INVARIANTS # Enable calls of extra sanity checking options INVARIANT_SUPPORT # Extra sanity checks of internal structures, required by INVARIANTS options WITNESS # Enable checks to detect deadlocks and cycles options WITNESS_SKIPSPIN# Don't run witness on spinlocks for speed options DIAGNOSTIC After kernel panic ddb prompt will be waiting for you. Type in: dump enter reset enter Make sure you have swap that can handle crashdumps. already did this part and debug part, but not DDB. As you see - hang not crashdump how much would it slow down whole thing? If less than 2 times it can be - CPU are rerely half loaded Are you asking about overhead of DDB or all debug options? I don't think that DDB support can be accounted for slowdown. As for the rest, it's hard to say. I guess it depends on your workload, also I never performed any benchmarks to compare this and I'm unaware of any. In other words, you have to test it yourself. -- Mateusz Guzik mjguzik gmail.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: reason for magic crashes.
Are you asking about overhead of DDB or all debug options? all. invariants, witness etc. I don't think that DDB support can be accounted for slowdown. As for the rest, it's hard to say. I guess it depends on your workload, also I never performed any benchmarks to compare this and I'm unaware of any. In other words, you have to test it yourself. we will see. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
