Re: Netconsole scripts for Debian
On Tue, 2014-06-17 at 13:30 +0400, Pavel Odintsov wrote: Hello, folks! I'm prepared and thoroughly tested scripts for managing netconsole kernel facility in Debian 6/7. Netconsole facility is very useful for debugging kernel bugs. These scripts are inspired by CentOS 6 netconsole scripts in some cases and work in same way. Scripts are written in clean bash. All scripts and configuration you can find here: https://github.com/FastVPSEestiOu/debian_netconsole Yes, it's possible to configure netconsole statically (i.e. as /etc/modprobe.d/netconsole.conf) but it's a bad way because it relies on network configuration (i.e. gateway address, server address) which may be changed and netconsole will not work. I tested both startup option: 1) As /etc/init.d/netconsole 2) As /etc/network/if-up.d/netconsole script IMHO, the first variant is more reliable and convenient for this task. But both variants have some troubles because some network cards (like e1000) call /etc/network/if-up.d and /etc/init.d scripts on network level BEFORE real network initialization. Network drivers do not control execution of ifupdown scripts at all. I think the problem you're seeing is that the physical link is down immediately after the interface is brought up. This should be expected because the link may have be turned off or limited to low speed to save power when the operating system is not using it. Bringing the link up then requires autonegotiation and link training, which *should* take less than a second but may sometimes take longer. I fixed this issue only by adding fixed timeout in my /etc/init.d/netconsole script. In the same way this issue was fixed in RH. Maybe you can provide more convenient solution? Thank you for any feedback! * You should detect the distribution using the lsb_release command rather than reading /etc/issue * Changing verbosity level should be optional * You should not assume that 'ip' is installed in /sbin; instead make sure that's included in $PATH * You should not assume that the destination server is reached by the default route; instead use 'ip route get $DESTINATION_SERVER_IP' to find the gateway and source address to use Ben. -- Ben Hutchings In a hierarchy, every employee tends to rise to his level of incompetence. signature.asc Description: This is a digitally signed message part
Re: Netconsole scripts for Debian
On Wed, Aug 27, 2014 at 11:11:53PM -0700, Ben Hutchings wrote: I think the problem you're seeing is that the physical link is down immediately after the interface is brought up. This should be expected because the link may have be turned off or limited to low speed to save power when the operating system is not using it. Bringing the link up then requires autonegotiation and link training, which *should* take less than a second but may sometimes take longer. Also, some Spanning Tree implementations keep a switch port in BLOCKED state for 30 seconds after a link state change to ensure that no loop was built. Greetings Marc -- - Marc Haber | I don't trust Computers. They | Mailadresse im Header Leimen, Germany| lose things.Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600420 -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140828073901.gb5...@torres.zugschlus.de
Re: Netconsole scripts for Debian
Hello! It's very important for debugging kernel oops when syslog over network running, because syslog can't send any information on dead kernel. When you haven't persistent kvm/idrac/ilo it's only one way for getting debug information about kernel hang. But why I need to integrate it with initrd? On Wed, Jun 18, 2014 at 10:19 AM, Ritesh Raj Sarraf r...@debian.org wrote: On 06/17/2014 03:00 PM, Pavel Odintsov wrote: Hello, folks! I'm prepared and thoroughly tested scripts for managing netconsole kernel facility in Debian 6/7. Netconsole facility is very useful for debugging kernel bugs. These scripts are inspired by CentOS 6 netconsole scripts in some cases and work in same way. Scripts are written in clean bash. All scripts and configuration you can find here: https://github.com/FastVPSEestiOu/debian_netconsole Yes, it's possible to configure netconsole statically (i.e. as /etc/modprobe.d/netconsole.conf) but it's a bad way because it relies on network configuration (i.e. gateway address, server address) which may be changed and netconsole will not work. I tested both startup option: 1) As /etc/init.d/netconsole 2) As /etc/network/if-up.d/netconsole script IMHO, the first variant is more reliable and convenient for this task. But both variants have some troubles because some network cards (like e1000) call /etc/network/if-up.d and /etc/init.d scripts on network level BEFORE real network initialization. I fixed this issue only by adding fixed timeout in my /etc/init.d/netconsole script. In the same way this issue was fixed in RH. Maybe you can provide more convenient solution? Thank you for any feedback! You should add support for it in initrd. netconsole is more useful at that stage. In real boot, why would one want netconsole if syslog is running. -- Given the large number of mailing lists I follow, I request you to CC me in replies for quicker response -- Sincerely yours, Pavel Odintsov -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/calgsdbdzydhbmmbilj5g-frn-ejv3rot+nlwmwuleaaans4...@mail.gmail.com
Re: Netconsole scripts for Debian
Hello! At now I configure netconsole dynamically and got network configuration from ip route command. For getting netconsole server MAC address I use arping to it. But I can generate modprobe configuration file with current configuration and integrate it to initrd. You can check my approach here: 1) On first netconsole script start I can get netconsole collector IP and MAC address (of destination server or gateway for current server) 2) I create /etc/modprobe.d/netconsole.conf configuration with fixed MAC, PORT and IP of netconsole server 3) I add netconsole to initramfs configuration file (/etc/initramfs-tools/modules) with fixed IP/MAC configuration 4) I rebuild initramfs for current kernel: update-initramfs -v -u -k `uname -r` -t 5) On every system run I will check if MAC/IP of destination server was changes I will reconfigure netconsole modprobe.conf file and rebuilt initrd. It's ok? On Wed, Jun 18, 2014 at 1:59 PM, Bjørn Mork bj...@mork.no wrote: Ritesh Raj Sarraf r...@debian.org writes: On 06/17/2014 03:00 PM, Pavel Odintsov wrote: Hello, folks! I'm prepared and thoroughly tested scripts for managing netconsole kernel facility in Debian 6/7. Netconsole facility is very useful for debugging kernel bugs. These scripts are inspired by CentOS 6 netconsole scripts in some cases and work in same way. Scripts are written in clean bash. All scripts and configuration you can find here: https://github.com/FastVPSEestiOu/debian_netconsole Yes, it's possible to configure netconsole statically (i.e. as /etc/modprobe.d/netconsole.conf) but it's a bad way because it relies on network configuration (i.e. gateway address, server address) which may be changed and netconsole will not work. I tested both startup option: 1) As /etc/init.d/netconsole 2) As /etc/network/if-up.d/netconsole script IMHO, the first variant is more reliable and convenient for this task. But both variants have some troubles because some network cards (like e1000) call /etc/network/if-up.d and /etc/init.d scripts on network level BEFORE real network initialization. I fixed this issue only by adding fixed timeout in my /etc/init.d/netconsole script. In the same way this issue was fixed in RH. Maybe you can provide more convenient solution? Thank you for any feedback! You should add support for it in initrd. netconsole is more useful at that stage. In real boot, why would one want netconsole if syslog is running. To debug a driver/kernel crash. netconsole is generally a very useful feature for oops debugging on laptops and other devices with no serial ports. Having some scripts making it simpler to configure sounds like a great idea. Will these depend on configfs being mounted? Or are they only using module parameters? Bjørn -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/87ppi6sfma@nemi.mork.no -- Sincerely yours, Pavel Odintsov -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/CALgsdbfxY5GDO8rKvMf0k40UqD+34NHFe8=kuo7cmtl-jjk...@mail.gmail.com
Re: Netconsole scripts for Debian
Pavel Odintsov pavel.odint...@gmail.com writes: Hello! At now I configure netconsole dynamically and got network configuration from ip route command. For getting netconsole server MAC address I use arping to it. But I can generate modprobe configuration file with current configuration and integrate it to initrd. You can check my approach here: 1) On first netconsole script start I can get netconsole collector IP and MAC address (of destination server or gateway for current server) The MAC address is optional and should be so in your package too. 2) I create /etc/modprobe.d/netconsole.conf configuration with fixed MAC, PORT and IP of netconsole server Why? You have to load the module from a script anyway, so you could just set the options there. 3) I add netconsole to initramfs configuration file (/etc/initramfs-tools/modules) with fixed IP/MAC configuration 4) I rebuild initramfs for current kernel: update-initramfs -v -u -k `uname -r` -t 5) On every system run I will check if MAC/IP of destination server was changes I will reconfigure netconsole modprobe.conf file and rebuilt initrd. There is already netconsole= cmdline support in initramfs-tools. Whatever you add should integrate nicely with this IMHO. Bjørn -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/87ionxq9vt@nemi.mork.no
Re: Netconsole scripts for Debian
On 06/19/2014 06:39 PM, Pavel Odintsov wrote: It's very important for debugging kernel oops when syslog over network running, because syslog can't send any information on dead kernel. When you haven't persistent kvm/idrac/ilo it's only one way for getting debug information about kernel hang. I don't know what you mean by a dead kernel here. If, say, the e1000e device driver is crashy, irrespective of what you use (netconsole or syslog), there's not much help. In this case, you'll then have to resort to serial console. But why I need to integrate it with initrd? Because it fills the gap in between kernel load to real init start. Imagine a scenario where as soon as the disk driver is loaded, the kernel panics. How would you debug that ? PS: It is just a suggestion. Your patch is still valid, just not very relevant to the use cases. -- Ritesh Raj Sarraf | http://people.debian.org/~rrs Debian - The Universal Operating System signature.asc Description: OpenPGP digital signature
Re: Netconsole scripts for Debian
On 06/17/2014 03:00 PM, Pavel Odintsov wrote: Hello, folks! I'm prepared and thoroughly tested scripts for managing netconsole kernel facility in Debian 6/7. Netconsole facility is very useful for debugging kernel bugs. These scripts are inspired by CentOS 6 netconsole scripts in some cases and work in same way. Scripts are written in clean bash. All scripts and configuration you can find here: https://github.com/FastVPSEestiOu/debian_netconsole Yes, it's possible to configure netconsole statically (i.e. as /etc/modprobe.d/netconsole.conf) but it's a bad way because it relies on network configuration (i.e. gateway address, server address) which may be changed and netconsole will not work. I tested both startup option: 1) As /etc/init.d/netconsole 2) As /etc/network/if-up.d/netconsole script IMHO, the first variant is more reliable and convenient for this task. But both variants have some troubles because some network cards (like e1000) call /etc/network/if-up.d and /etc/init.d scripts on network level BEFORE real network initialization. I fixed this issue only by adding fixed timeout in my /etc/init.d/netconsole script. In the same way this issue was fixed in RH. Maybe you can provide more convenient solution? Thank you for any feedback! You should add support for it in initrd. netconsole is more useful at that stage. In real boot, why would one want netconsole if syslog is running. -- Given the large number of mailing lists I follow, I request you to CC me in replies for quicker response signature.asc Description: OpenPGP digital signature
Re: Netconsole scripts for Debian
Ritesh Raj Sarraf r...@debian.org writes: On 06/17/2014 03:00 PM, Pavel Odintsov wrote: Hello, folks! I'm prepared and thoroughly tested scripts for managing netconsole kernel facility in Debian 6/7. Netconsole facility is very useful for debugging kernel bugs. These scripts are inspired by CentOS 6 netconsole scripts in some cases and work in same way. Scripts are written in clean bash. All scripts and configuration you can find here: https://github.com/FastVPSEestiOu/debian_netconsole Yes, it's possible to configure netconsole statically (i.e. as /etc/modprobe.d/netconsole.conf) but it's a bad way because it relies on network configuration (i.e. gateway address, server address) which may be changed and netconsole will not work. I tested both startup option: 1) As /etc/init.d/netconsole 2) As /etc/network/if-up.d/netconsole script IMHO, the first variant is more reliable and convenient for this task. But both variants have some troubles because some network cards (like e1000) call /etc/network/if-up.d and /etc/init.d scripts on network level BEFORE real network initialization. I fixed this issue only by adding fixed timeout in my /etc/init.d/netconsole script. In the same way this issue was fixed in RH. Maybe you can provide more convenient solution? Thank you for any feedback! You should add support for it in initrd. netconsole is more useful at that stage. In real boot, why would one want netconsole if syslog is running. To debug a driver/kernel crash. netconsole is generally a very useful feature for oops debugging on laptops and other devices with no serial ports. Having some scripts making it simpler to configure sounds like a great idea. Will these depend on configfs being mounted? Or are they only using module parameters? Bjørn -- To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/87ppi6sfma@nemi.mork.no
