Re: Failsafe on kernel panic
On 1/20/13 6:07 PM, Willem Jan Withagen wrote: > On 17-1-2013 4:18, Ian Lepore wrote: >> On Wed, 2013-01-16 at 23:27 +0200, Sami Halabi wrote: >>> Thank you for your response, very helpful. >>> one question - how do i configure auto-reboot once kernel panic occurs? >>> >>> Sami >>> >> >> From src/sys/conf/NOTES, this may be what you're looking for... >> >> # >> # Don't enter the debugger for a panic. Intended for unattended operation >> # where you may want to enter the debugger from the console, but still want >> # the machine to recover from a panic. >> # >> options KDB_UNATTENDED >> >> But I think it only has meaning if you have option KDB in effect, >> otherwise it should just reboot itself after a 15 second pause. > > Well it is not the magical fix-all solution. > > Last night I had to drive to the colo (lucky for me a 5 min drive.) > because I could not get a system to reboot/recover from a crash. > > Upon arrival the system was crashed and halted on the message: > rebooting in 15 sec. > I've seen the same thing many and many times. Now I'm using ddb to save crash dump and reboot machine on panic. It's much more reliable. -- Andrey Zonov signature.asc Description: OpenPGP digital signature
Re: Failsafe on kernel panic
I started investigating ipmi, so far i can configure IP from fbsd to ipmi. My question is how to access it? Can it be done inband attached to one oc the Ibm nics kn the board? or knlh out oc band? In case of oob any knows if the iLO plug is pure rj45 in ibm servers (specially x3250/3550)? Thanks in advance Sami בתאריך 20 בינו 2013 16:07, מאת "Willem Jan Withagen" : > On 17-1-2013 4:18, Ian Lepore wrote: > > On Wed, 2013-01-16 at 23:27 +0200, Sami Halabi wrote: > >> Thank you for your response, very helpful. > >> one question - how do i configure auto-reboot once kernel panic occurs? > >> > >> Sami > >> > > > > From src/sys/conf/NOTES, this may be what you're looking for... > > > > # > > # Don't enter the debugger for a panic. Intended for unattended operation > > # where you may want to enter the debugger from the console, but still > want > > # the machine to recover from a panic. > > # > > options KDB_UNATTENDED > > > > But I think it only has meaning if you have option KDB in effect, > > otherwise it should just reboot itself after a 15 second pause. > > Well it is not the magical fix-all solution. > > Last night I had to drive to the colo (lucky for me a 5 min drive.) > because I could not get a system to reboot/recover from a crash. > > Upon arrival the system was crashed and halted on the message: > rebooting in 15 sec. > > Which but those 15 secs are would have gone by for about 10-20 minutes. > fysically rebooting or resetting ended up in the same position: > rebooting in 15 sec. > Without ever getting to actually rebooting. > > So if I (you) have servers 2 hours away, I usually try to work on > upgrading/rebooting during business hours. And remote hands can get me > out of trouble > > IPMI is another nice way of getting at the server in these cases. But > that requires a lot more infra and tinkering. > > --WjW > > > > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Failsafe on kernel panic
On 17-1-2013 4:18, Ian Lepore wrote: > On Wed, 2013-01-16 at 23:27 +0200, Sami Halabi wrote: >> Thank you for your response, very helpful. >> one question - how do i configure auto-reboot once kernel panic occurs? >> >> Sami >> > > From src/sys/conf/NOTES, this may be what you're looking for... > > # > # Don't enter the debugger for a panic. Intended for unattended operation > # where you may want to enter the debugger from the console, but still want > # the machine to recover from a panic. > # > options KDB_UNATTENDED > > But I think it only has meaning if you have option KDB in effect, > otherwise it should just reboot itself after a 15 second pause. Well it is not the magical fix-all solution. Last night I had to drive to the colo (lucky for me a 5 min drive.) because I could not get a system to reboot/recover from a crash. Upon arrival the system was crashed and halted on the message: rebooting in 15 sec. Which but those 15 secs are would have gone by for about 10-20 minutes. fysically rebooting or resetting ended up in the same position: rebooting in 15 sec. Without ever getting to actually rebooting. So if I (you) have servers 2 hours away, I usually try to work on upgrading/rebooting during business hours. And remote hands can get me out of trouble IPMI is another nice way of getting at the server in these cases. But that requires a lot more infra and tinkering. --WjW ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Failsafe on kernel panic
On Wednesday, January 16, 2013 4:27:53 pm Sami Halabi wrote: > Thank you for your response, very helpful. > one question - how do i configure auto-reboot once kernel panic occurs? Unless you've added DDB and KDB to your kernel it will reboot by default on a panic. Stable kernel configs also include the unattended option so that even with the debugger present they reboot by default on a panic. -- John Baldwin ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Failsafe on kernel panic
On Thu, Jan 17, 2013 at 04:14:21PM +0200, Sami Halabi wrote: > Hi, > Upon panic no auto restart occurs. > How do I know/activate these watchdogs? > Sami You can try starting watchdogd with 'service watchdogd onestart', and have it automatically start during boot by adding 'watchdogd_enable="YES"' to rc.conf. You can test by starting watchdogd and sending SIGKILL to it - if everything's working properly, the system should reboot after the timeout period (16s by default). If you don't have a hardware watchdog (or have one that isn't supported by any drivers), watchdogd will fail to start. -Mark > בתאריך 17 בינו 2013 15:35, מאת "Ian Lepore" : > > > On Thu, 2013-01-17 at 08:38 +0200, Sami Halabi wrote: > > > btw: i don't see any options in my kernel config for KBD / Unatteneded , > > th > > > eonly thing that mention its > > > is: device ukbd > > > > > > Sami > > > > I think if you don't have any kdb options turned on, then a panic should > > automatically store a crashdump to swap, then reboot the machine. If > > that's not working, perhaps it locks up trying to store the dump? > > > > If the hardware has a watchdog timer, enabling that might be the best > > way to ensure a reboot on any kind of crash or hang. > > > > -- Ian > > > > > > > On Thu, Jan 17, 2013 at 6:45 AM, Sami Halabi wrote: > > > > > > > Its only a kernel option? There is no flag to pass to the loader? > > > > > > > > SAMI > > <> 17 2013 05:18, "Ian Lepore" : > > > > > > > > On Wed, 2013-01-16 at 23:27 +0200, Sami Halabi wrote: > > > >> > Thank you for your response, very helpful. > > > >> > one question - how do i configure auto-reboot once kernel panic > > occurs? > > > >> > > > > >> > Sami > > > >> > > > > >> > > > >> From src/sys/conf/NOTES, this may be what you're looking for... > > > >> > > > >> # > > > >> # Don't enter the debugger for a panic. Intended for unattended > > operation > > > >> # where you may want to enter the debugger from the console, but still > > > >> want > > > >> # the machine to recover from a panic. > > > >> # > > > >> options KDB_UNATTENDED > > > >> > > > >> But I think it only has meaning if you have option KDB in effect, > > > >> otherwise it should just reboot itself after a 15 second pause. > > > >> > > > >> -- Ian > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > > >> > On Wed, Jan 16, 2013 at 10:13 PM, John Baldwin > > wrote: > > > >> > > > > >> > > On Wednesday, January 16, 2013 2:25:33 pm Sami Halabi wrote: > > > >> > > > Hi everyone, > > > >> > > > I have a production box, in which I want to install new kernel > > > >> without > > > >> > > any > > > >> > > > remotd kvn. > > > >> > > > my problem is its 2 hours away, and if a kernel panic occurs I > > got a > > > >> > > > problem. > > > >> > > > I woner if I can seg failsafe script to load the old kernel in > > case > > > >> of > > > >> > > > psnic. > > > >> > > > > > >> > > man nextboot (if you are using UFS) > > > >> > > > > > >> > > -- > > > >> > > John Baldwin > > > >> > > > > > >> > > > > >> > > > > >> > > > > >> > > > >> > > > >> > > > > > > > > > -- > > > Sami Halabi > > > Information Systems Engineer > > > NMS Projects Expert > > > FreeBSD SysAdmin Expert > > > ___ > > > freebsd-hackers@freebsd.org mailing list > > > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > > To unsubscribe, send any mail to " > > freebsd-hackers-unsubscr...@freebsd.org" > > > > > > > ___ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org" ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Failsafe on kernel panic
Hi, Upon panic no auto restart occurs. How do I know/activate these watchdogs? Sami בתאריך 17 בינו 2013 15:35, מאת "Ian Lepore" : > On Thu, 2013-01-17 at 08:38 +0200, Sami Halabi wrote: > > btw: i don't see any options in my kernel config for KBD / Unatteneded , > th > > eonly thing that mention its > > is: device ukbd > > > > Sami > > I think if you don't have any kdb options turned on, then a panic should > automatically store a crashdump to swap, then reboot the machine. If > that's not working, perhaps it locks up trying to store the dump? > > If the hardware has a watchdog timer, enabling that might be the best > way to ensure a reboot on any kind of crash or hang. > > -- Ian > > > > On Thu, Jan 17, 2013 at 6:45 AM, Sami Halabi wrote: > > > > > Its only a kernel option? There is no flag to pass to the loader? > > > > > > SAMI > <> 17 2013 05:18, "Ian Lepore" : > > > > > > On Wed, 2013-01-16 at 23:27 +0200, Sami Halabi wrote: > > >> > Thank you for your response, very helpful. > > >> > one question - how do i configure auto-reboot once kernel panic > occurs? > > >> > > > >> > Sami > > >> > > > >> > > >> From src/sys/conf/NOTES, this may be what you're looking for... > > >> > > >> # > > >> # Don't enter the debugger for a panic. Intended for unattended > operation > > >> # where you may want to enter the debugger from the console, but still > > >> want > > >> # the machine to recover from a panic. > > >> # > > >> options KDB_UNATTENDED > > >> > > >> But I think it only has meaning if you have option KDB in effect, > > >> otherwise it should just reboot itself after a 15 second pause. > > >> > > >> -- Ian > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > > >> > On Wed, Jan 16, 2013 at 10:13 PM, John Baldwin > wrote: > > >> > > > >> > > On Wednesday, January 16, 2013 2:25:33 pm Sami Halabi wrote: > > >> > > > Hi everyone, > > >> > > > I have a production box, in which I want to install new kernel > > >> without > > >> > > any > > >> > > > remotd kvn. > > >> > > > my problem is its 2 hours away, and if a kernel panic occurs I > got a > > >> > > > problem. > > >> > > > I woner if I can seg failsafe script to load the old kernel in > case > > >> of > > >> > > > psnic. > > >> > > > > >> > > man nextboot (if you are using UFS) > > >> > > > > >> > > -- > > >> > > John Baldwin > > >> > > > > >> > > > >> > > > >> > > > >> > > >> > > >> > > > > > > -- > > Sami Halabi > > Information Systems Engineer > > NMS Projects Expert > > FreeBSD SysAdmin Expert > > ___ > > freebsd-hackers@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to " > freebsd-hackers-unsubscr...@freebsd.org" > > > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Failsafe on kernel panic
On Thu, 2013-01-17 at 08:38 +0200, Sami Halabi wrote: > btw: i don't see any options in my kernel config for KBD / Unatteneded , th > eonly thing that mention its > is: device ukbd > > Sami I think if you don't have any kdb options turned on, then a panic should automatically store a crashdump to swap, then reboot the machine. If that's not working, perhaps it locks up trying to store the dump? If the hardware has a watchdog timer, enabling that might be the best way to ensure a reboot on any kind of crash or hang. -- Ian > On Thu, Jan 17, 2013 at 6:45 AM, Sami Halabi wrote: > > > Its only a kernel option? There is no flag to pass to the loader? > > > > SAMI <> 17 2013 05:18, "Ian Lepore" : > > > > On Wed, 2013-01-16 at 23:27 +0200, Sami Halabi wrote: > >> > Thank you for your response, very helpful. > >> > one question - how do i configure auto-reboot once kernel panic occurs? > >> > > >> > Sami > >> > > >> > >> From src/sys/conf/NOTES, this may be what you're looking for... > >> > >> # > >> # Don't enter the debugger for a panic. Intended for unattended operation > >> # where you may want to enter the debugger from the console, but still > >> want > >> # the machine to recover from a panic. > >> # > >> options KDB_UNATTENDED > >> > >> But I think it only has meaning if you have option KDB in effect, > >> otherwise it should just reboot itself after a 15 second pause. > >> > >> -- Ian > >> > >> > >> > >> > >> > >> > >> > > >> > On Wed, Jan 16, 2013 at 10:13 PM, John Baldwin wrote: > >> > > >> > > On Wednesday, January 16, 2013 2:25:33 pm Sami Halabi wrote: > >> > > > Hi everyone, > >> > > > I have a production box, in which I want to install new kernel > >> without > >> > > any > >> > > > remotd kvn. > >> > > > my problem is its 2 hours away, and if a kernel panic occurs I got a > >> > > > problem. > >> > > > I woner if I can seg failsafe script to load the old kernel in case > >> of > >> > > > psnic. > >> > > > >> > > man nextboot (if you are using UFS) > >> > > > >> > > -- > >> > > John Baldwin > >> > > > >> > > >> > > >> > > >> > >> > >> > > > -- > Sami Halabi > Information Systems Engineer > NMS Projects Expert > FreeBSD SysAdmin Expert > ___ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org" ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Failsafe on kernel panic
btw: i don't see any options in my kernel config for KBD / Unatteneded , th eonly thing that mention its is: device ukbd Sami On Thu, Jan 17, 2013 at 6:45 AM, Sami Halabi wrote: > Its only a kernel option? There is no flag to pass to the loader? > > SAMI > בתאריך 17 בינו 2013 05:18, מאת "Ian Lepore" : > > On Wed, 2013-01-16 at 23:27 +0200, Sami Halabi wrote: >> > Thank you for your response, very helpful. >> > one question - how do i configure auto-reboot once kernel panic occurs? >> > >> > Sami >> > >> >> From src/sys/conf/NOTES, this may be what you're looking for... >> >> # >> # Don't enter the debugger for a panic. Intended for unattended operation >> # where you may want to enter the debugger from the console, but still >> want >> # the machine to recover from a panic. >> # >> options KDB_UNATTENDED >> >> But I think it only has meaning if you have option KDB in effect, >> otherwise it should just reboot itself after a 15 second pause. >> >> -- Ian >> >> >> >> >> >> >> > >> > On Wed, Jan 16, 2013 at 10:13 PM, John Baldwin wrote: >> > >> > > On Wednesday, January 16, 2013 2:25:33 pm Sami Halabi wrote: >> > > > Hi everyone, >> > > > I have a production box, in which I want to install new kernel >> without >> > > any >> > > > remotd kvn. >> > > > my problem is its 2 hours away, and if a kernel panic occurs I got a >> > > > problem. >> > > > I woner if I can seg failsafe script to load the old kernel in case >> of >> > > > psnic. >> > > >> > > man nextboot (if you are using UFS) >> > > >> > > -- >> > > John Baldwin >> > > >> > >> > >> > >> >> >> -- Sami Halabi Information Systems Engineer NMS Projects Expert FreeBSD SysAdmin Expert ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Failsafe on kernel panic
Its only a kernel option? There is no flag to pass to the loader? SAMI בתאריך 17 בינו 2013 05:18, מאת "Ian Lepore" : > On Wed, 2013-01-16 at 23:27 +0200, Sami Halabi wrote: > > Thank you for your response, very helpful. > > one question - how do i configure auto-reboot once kernel panic occurs? > > > > Sami > > > > From src/sys/conf/NOTES, this may be what you're looking for... > > # > # Don't enter the debugger for a panic. Intended for unattended operation > # where you may want to enter the debugger from the console, but still want > # the machine to recover from a panic. > # > options KDB_UNATTENDED > > But I think it only has meaning if you have option KDB in effect, > otherwise it should just reboot itself after a 15 second pause. > > -- Ian > > > > > > > > > > On Wed, Jan 16, 2013 at 10:13 PM, John Baldwin wrote: > > > > > On Wednesday, January 16, 2013 2:25:33 pm Sami Halabi wrote: > > > > Hi everyone, > > > > I have a production box, in which I want to install new kernel > without > > > any > > > > remotd kvn. > > > > my problem is its 2 hours away, and if a kernel panic occurs I got a > > > > problem. > > > > I woner if I can seg failsafe script to load the old kernel in case > of > > > > psnic. > > > > > > man nextboot (if you are using UFS) > > > > > > -- > > > John Baldwin > > > > > > > > > > > > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Failsafe on kernel panic
On Wed, 2013-01-16 at 23:27 +0200, Sami Halabi wrote: > Thank you for your response, very helpful. > one question - how do i configure auto-reboot once kernel panic occurs? > > Sami > >From src/sys/conf/NOTES, this may be what you're looking for... # # Don't enter the debugger for a panic. Intended for unattended operation # where you may want to enter the debugger from the console, but still want # the machine to recover from a panic. # options KDB_UNATTENDED But I think it only has meaning if you have option KDB in effect, otherwise it should just reboot itself after a 15 second pause. -- Ian > > On Wed, Jan 16, 2013 at 10:13 PM, John Baldwin wrote: > > > On Wednesday, January 16, 2013 2:25:33 pm Sami Halabi wrote: > > > Hi everyone, > > > I have a production box, in which I want to install new kernel without > > any > > > remotd kvn. > > > my problem is its 2 hours away, and if a kernel panic occurs I got a > > > problem. > > > I woner if I can seg failsafe script to load the old kernel in case of > > > psnic. > > > > man nextboot (if you are using UFS) > > > > -- > > John Baldwin > > > > > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Failsafe on kernel panic
Thank you for your response, very helpful. one question - how do i configure auto-reboot once kernel panic occurs? Sami On Wed, Jan 16, 2013 at 10:13 PM, John Baldwin wrote: > On Wednesday, January 16, 2013 2:25:33 pm Sami Halabi wrote: > > Hi everyone, > > I have a production box, in which I want to install new kernel without > any > > remotd kvn. > > my problem is its 2 hours away, and if a kernel panic occurs I got a > > problem. > > I woner if I can seg failsafe script to load the old kernel in case of > > psnic. > > man nextboot (if you are using UFS) > > -- > John Baldwin > -- Sami Halabi Information Systems Engineer NMS Projects Expert FreeBSD SysAdmin Expert ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Failsafe on kernel panic
On Wednesday, January 16, 2013 2:25:33 pm Sami Halabi wrote: > Hi everyone, > I have a production box, in which I want to install new kernel without any > remotd kvn. > my problem is its 2 hours away, and if a kernel panic occurs I got a > problem. > I woner if I can seg failsafe script to load the old kernel in case of > psnic. man nextboot (if you are using UFS) -- John Baldwin ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
