Re: [RFC] autokdump - automated kdump testsuite
- Original Message - > From: "Vivek Goyal" > To: "CAI Qian" > Cc: "ltp-list" sourceforge.net>, "kexec kdump redhat mailing list" > , > "kexec" , "linux-kernel" vger.kernel.org>, > "crash-utility" > redhat.com> > Sent: Monday, September 22, 2014 10:47:13 PM > Subject: Re: [RFC] autokdump - automated kdump testsuite > > On Mon, Sep 22, 2014 at 09:00:00AM -0400, CAI Qian wrote: > > > > > > - Original Message - > > > From: "Vivek Goyal" > > > To: "CAI Qian" > > > Cc: "linux-kernel" vger.kernel.org>, "ltp-list" > > > sourceforge.net>, "crash-utility" > > > redhat.com>, "kexec" , "kexec > > > kdump redhat mailing list" > > > > > > Sent: Friday, September 19, 2014 9:22:36 PM > > > Subject: Re: [RFC] autokdump - automated kdump testsuite > > > > > > On Fri, Sep 19, 2014 at 05:52:25AM -0400, CAI Qian wrote: > > > > I plan to release an automated kdump testsuite that will be > > > > > > So will this be a standalone test suit? Can it be merged with > > > something already existing say, LTP. > > Yes, it is likely to be standalone. It won't make use of the LTP > > API, and the LTP kdump test suite is outdated, so there is no > > benefit to continue working over there. > > So why make it standalone and not replace the old LTP kdump test suite > with this new one? It will be totally a rewrite from scratch, and will have no direct relationship with the rest of the LTP. > > > > > > > > focus on testing kernel and the crash utility. It should work > > > > for all major distros since it will use none of distro-specific > > > > stuff, and also support different arches including x86, ARM, > > > > PPC64 and s390x. > > > > > > > > It does the following: > > > > 1) check if there is a memory reserved for kdump. If not, > > > >reserve the memory and reboot the system. > > > > 2) once the system is back, load kexec on panic and > > > >prepare a separate initramfs that including needed > > > >modules to load a local filesystem and necessary utilities > > > > > > So you will write logic to prepare custom initramfs or will rely > > > on dracut or some other utility for that. > > I'll probably prepare custom initramfs for the sake of simplicity. > > Well, preparing custom initramfs will become very tricky. We used > to do that and finally we switched to dracut. > > Why not simply let the respective service on the host do this job and > test only makes sure that kdump service is running. It feels little > out of place that a test is generating custom initramfs. Because not every distro will have a kdump service like Fedora. > > > > > > > >in order to analyse /proc/vmcore in the 2nd kernel. > > > > 3) trigger the system crash using methods like sysrq-c, NMI, > > > >and panic_on_hung_task etc. > > > > 4) in the 2nd kernel, mount a filesystem and use the crash > > > >utility to analyse /proc/vmcore. Then, gather the analyse > > > >logs, serial console output, dmesg etc into the filesystem. > > > > > > Why not save core and boot back in first kernel and then analyze. > > > > > > Trying to work directly with /proc/vmcore does not test makedumfile > > > which everybody uses. Also it will require more memory to be reserved > > > and packing crash and debug vmlinux into initramfs. > > The additional memory for vmlinux and the crash utility is predictable > > and manageable, so it can just ask 256M memory reserved before running > > the program. On the other hand, it is not usually feasible to ask > > the systems under testing has enough available disk spaces bigger than > > the memory size. > > makedumpfile will reduce the vmcore file size to few hundreds of mega > bytes on most of the systems. Especially, this is just a test, so > system will be lightly loaded and vmcore will be small after filtering. It probably actually have test cases to heavily loaded the memory before dumping. > > If there is not enough space, test fails, period. I don't think there > is any need to try to circumvent that and try to run crash in initramfs. > And in the process we don't test makedumpfile which is very imporatnt > component of this whole process. Test makedumpfile is in plan. Tests failed because of lack of disk space is a testsuite design problem, and especially problematic on those large memory systems that we had seen more and more those days. > > IMHO, just rely on "systemctl start kdump" to generate and load custom > initramfs and save filtered vmcore to root fs by default and alanyze > vmcore post reboot. That will keep things simple. Again, not every major distro has that. CAI Qian > > Thanks > Vivek > > ___ > kexec mailing list > kexec@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/kexec > ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [RFC] autokdump - automated kdump testsuite
- Original Message - > From: "Vivek Goyal" > To: "CAI Qian" > Cc: "linux-kernel" , "ltp-list" > , "crash-utility" > , "kexec" , "kexec kdump > redhat mailing list" > > Sent: Friday, September 19, 2014 9:22:36 PM > Subject: Re: [RFC] autokdump - automated kdump testsuite > > On Fri, Sep 19, 2014 at 05:52:25AM -0400, CAI Qian wrote: > > I plan to release an automated kdump testsuite that will be > > So will this be a standalone test suit? Can it be merged with > something already existing say, LTP. Yes, it is likely to be standalone. It won't make use of the LTP API, and the LTP kdump test suite is outdated, so there is no benefit to continue working over there. > > > focus on testing kernel and the crash utility. It should work > > for all major distros since it will use none of distro-specific > > stuff, and also support different arches including x86, ARM, > > PPC64 and s390x. > > > > It does the following: > > 1) check if there is a memory reserved for kdump. If not, > >reserve the memory and reboot the system. > > 2) once the system is back, load kexec on panic and > >prepare a separate initramfs that including needed > >modules to load a local filesystem and necessary utilities > > So you will write logic to prepare custom initramfs or will rely > on dracut or some other utility for that. I'll probably prepare custom initramfs for the sake of simplicity. > > >in order to analyse /proc/vmcore in the 2nd kernel. > > 3) trigger the system crash using methods like sysrq-c, NMI, > >and panic_on_hung_task etc. > > 4) in the 2nd kernel, mount a filesystem and use the crash > >utility to analyse /proc/vmcore. Then, gather the analyse > >logs, serial console output, dmesg etc into the filesystem. > > Why not save core and boot back in first kernel and then analyze. > > Trying to work directly with /proc/vmcore does not test makedumfile > which everybody uses. Also it will require more memory to be reserved > and packing crash and debug vmlinux into initramfs. The additional memory for vmlinux and the crash utility is predictable and manageable, so it can just ask 256M memory reserved before running the program. On the other hand, it is not usually feasible to ask the systems under testing has enough available disk spaces bigger than the memory size. CAI Qian > > I think being able to test makedumpfile also is the key here. > > Thanks > Vivek > > 5) reboot back into the 1st kernel. > > > > implementation: > > It will setup a daemon to handle reboots. > > > > plan: > > I might also to test the makedumpfile all together later. > >CAI Qian > ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
[RFC] autokdump - automated kdump testsuite
I plan to release an automated kdump testsuite that will be focus on testing kernel and the crash utility. It should work for all major distros since it will use none of distro-specific stuff, and also support different arches including x86, ARM, PPC64 and s390x. It does the following: 1) check if there is a memory reserved for kdump. If not, reserve the memory and reboot the system. 2) once the system is back, load kexec on panic and prepare a separate initramfs that including needed modules to load a local filesystem and necessary utilities in order to analyse /proc/vmcore in the 2nd kernel. 3) trigger the system crash using methods like sysrq-c, NMI, and panic_on_hung_task etc. 4) in the 2nd kernel, mount a filesystem and use the crash utility to analyse /proc/vmcore. Then, gather the analyse logs, serial console output, dmesg etc into the filesystem. 5) reboot back into the 1st kernel. implementation: It will setup a daemon to handle reboots. plan: I might also to test the makedumpfile all together later. CAI Qian ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: failed to compile kexec from git
If I revert this commit, 6df15d1cca55b2cd59c79aed69f004e1b2a6ba36 kexec x86: drop truncation warning for crash kernel Everything is fine again. CAI Qian - Original Message - > From: "CAI Qian" > To: "kexec" > Sent: Wednesday, March 13, 2013 10:52:52 AM > Subject: failed to compile kexec from git > > Looks like this is broken recently. I cloned the tree > at the top commit been, > > e73d481c4add17ee953c80bb4be72372d90d93d3 > ppc/uImage: Add support for RAM Disks > > Then, error out. > > kexec/arch/i386/x86-linux-setup.c: In function > ‘setup_linux_system_parameters’: > kexec/arch/i386/x86-linux-setup.c:511:9: error: ‘kexec_flags’ > undeclared (first use in this function) > kexec/arch/i386/x86-linux-setup.c:511:9: note: each undeclared > identifier is reported only once for each function it appears in > make: *** [kexec/arch/i386/x86-linux-setup.o] Error 1 > > CAI Qian > > ___ > kexec mailing list > kexec@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/kexec > ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
failed to compile kexec from git
Looks like this is broken recently. I cloned the tree at the top commit been, e73d481c4add17ee953c80bb4be72372d90d93d3 ppc/uImage: Add support for RAM Disks Then, error out. kexec/arch/i386/x86-linux-setup.c: In function ‘setup_linux_system_parameters’: kexec/arch/i386/x86-linux-setup.c:511:9: error: ‘kexec_flags’ undeclared (first use in this function) kexec/arch/i386/x86-linux-setup.c:511:9: note: each undeclared identifier is reported only once for each function it appears in make: *** [kexec/arch/i386/x86-linux-setup.o] Error 1 CAI Qian ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: 3.9-rc1: crash kernel panic - not syncing: Can not allocate SWIOTLB buffer earlier and can't now provide you with the DMA bounce buffer
CC'ing kexec ML. Also mentioned that 3.8 has no such issue. This message looks suspicious and out of range while 3.8 reservation looks within the range. [0.00] Reserving 128MB of memory at 5216MB for crashkernel (System RAM: 3977MB) Wondering if anything to do with memblock again... CAI Qian - Original Message - > From: "WANG Chao" > To: "LKML" vger.kernel.org> > Cc: "CAI Qian" > Sent: Friday, March 8, 2013 1:54:37 PM > Subject: 3.9-rc1: crash kernel panic - not syncing: Can not allocate SWIOTLB > buffer earlier and can't now provide you > with the DMA bounce buffer > > Hi, All > > On 3.9-rc1, I load crash kernel with latest kexec-tools(up to > 28d413a), but > 2nd kernel panic at early time: > [2.948076] Kernel panic - not syncing: Can not allocate SWIOTLB > buffer earlier and can't now provide you with the DMA bounce buffer > [2.959958] Pid: 53, comm: khubd Not tainted 3.9.0-rc1+ #1 > [2.965426] Call Trace: > [2.967866] [] panic+0xc1/0x1d0 > [2.972644] [] > swiotlb_tbl_map_single+0x27c/0x280 > [2.978991] [] map_single+0x19/0x20 > [2.984115] [] swiotlb_map_page+0x6e/0x160 > [2.989845] [] > usb_hcd_map_urb_for_dma+0x230/0x4a0 > [2.996268] [] usb_hcd_submit_urb+0x295/0x8e0 > [3.002258] [] ? __dequeue_entity+0x2f/0x50 > [3.008076] [] ? __switch_to+0x13e/0x4a0 > [3.013632] [] usb_submit_urb+0xff/0x3d0 > [3.019186] [] ? __schedule+0x3de/0x7e0 > [3.024657] [] usb_start_wait_urb+0x6a/0x160 > [3.030560] [] ? __kmalloc+0x55/0x210 > [3.035856] [] ? usb_alloc_urb+0x1e/0x50 > [3.041411] [] usb_control_msg+0xde/0x140 > [3.047056] [] ? hub_port_init+0x310/0xaf0 > [3.052785] [] ? hub_port_init+0x2eb/0xaf0 > [3.058515] [] hub_port_init+0x338/0xaf0 > [3.064071] [] ? update_autosuspend+0x39/0x60 > [3.070062] [] ? > pm_runtime_set_autosuspend_delay+0x49/0x70 > [3.077264] [] > hub_port_connect_change+0x24a/0xaa0 > [3.083684] [] hub_events+0x2ea/0x910 > [3.088981] [] ? __schedule+0x3de/0x7e0 > [3.094451] [] hub_thread+0x35/0x1e0 > [3.099661] [] ? wake_up_bit+0x40/0x40 > [3.105045] [] ? hub_events+0x910/0x910 > [3.110514] [] kthread+0xc0/0xd0 > [3.115378] [] ? > kthread_create_on_node+0x120/0x120 > [3.121887] [] ret_from_fork+0x7c/0xb0 > [3.127271] [] ? > kthread_create_on_node+0x120/0x120 > > > Here's the full log: > # grep 'Crash' /proc/iomem > 14600-14dff : Crash kernel > > # dmesg | grep -i reserving > [0.00] Reserving 128MB of memory at 5216MB for crashkernel > (System RAM: 3977MB) > > # kexec -p /boot/vmlinuz-3.9.0-rc1+ > --command-line='console=ttyS0,115200n81' > # echo c > /proc/sysrq-trigger > > [ 217.879315] SysRq : Trigger a crash > [ 217.882836] BUG: unable to handle kernel NULL pointer dereference > at (null) > [ 217.890674] IP: [] sysrq_handle_crash+0x16/0x20 > [ 217.896773] PGD 13df22067 PUD 139726067 PMD 0 > [ 217.901244] Oops: 0002 [#1] SMP > [ 217.904491] Modules linked in: lockd sunrpc > nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE > ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 > iptable_nat nf_nat_ip > v4 nf_nat iptable_mangle ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 > xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter > ip6_tables iptable_filter ip_tables sg coretemp kvm_inte > l kvm e1000e iTCO_wdt crc32c_intel iTCO_vendor_support ptp > ghash_clmulni_intel pps_core mei microcode pcspkr i2c_i801 lpc_ich > mfd_core xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif i915 i2c_al > go_bit drm_kms_helper drm ahci libahci libata i2c_core video > dm_mirror dm_region_hash dm_log dm_mod > [ 217.963690] CPU 0 > [ 217.965526] Pid: 1206, comm: bash Not tainted 3.9.0-rc1+ #1 Intel > Corporation 2012 Client Platform/Emerald Lake 2 > [ 217.975948] RIP: 0010:[] [] > sysrq_handle_crash+0x16/0x20 > [ 217.984468] RSP: 0018:8801367e9e38 EFLAGS: 00010092 > [ 217.989765] RAX: 000f RBX: 819b67c0 RCX: > 88014e20ffe8 > [ 217.996881] RDX: RSI: 88014e20e3b8 RDI: > 0063 > [ 218.003998] RBP: 8801367e9e38 R08: 81c06280 R09: > 0419 > [ 218.03] R10: 0002 R11: 0418 R12: > 0063 > [ 218.018230] R13: 0286 R14: R15: > 0007 > [ 218.025346] FS: 7fdd48ace740() GS:88014e20() > knlGS: > [ 218.033416] CS: 0010 DS: ES: CR0: 80050033 > [ 218.039147] CR2:
Kdump failed with Intel IOMMU (WAS Re: kdump regression related to usb3 hub update)
Sorry, this has nothing to do with usb3 hub update. I made a mistake that passed amd_iommu=on instead intel_iommu=on to the successful base kernel run. Hence, it passed once there and all testing for other kernels used the right parameters were failed. The problem is still there though. Since this is the first time test this setup, not sure if this is regression or not. CAI Qian - Original Message - > Hello Sarah and others, > > There is a regression using the latest Sarah's xhci hubs-v3-rebase > tree, kdump kernel reset itself (log at the end of email). The first > kernel also setup using IOMMU (intel_iommu=on). Removed intel_iommu=on > from the kdump kernel parameter alone did not solve the problem. This > is only reproducible when the first kernel is using IOMMU. > > My environment setup is: > > PCI-E card -> USB3 Disk > -> USB3 Hub -> USB3 Disk > -> USB2 Hub -> USB2 Disk > > I have been identified one or a few of the following patches caused > the problem but need more time to narrow it down. > > 288ead45fa6637e959015d055304f521cbbc0575 > USB: remove bogus USB_PORT_FEAT_*_SPEED symbols > > 0eadcc09203349b11ca477ec367079b23d32ab91 > usb: USB3.0 ch11 definitions > > b1a72b70bbb82237b1041c78edee305503f6da31 > USB: Remove bitmap #define from hcd.h > > 50e6c947c6340f7ba81deca99e8a37f458a00e49 > USB: Fix usb_add_hcd() checkpatch errors. > > 0abde136582234dd720270ec3d5aeba69eae4d03 > xhci: Rework port suspend structures for limited ports. > > 8c0d6f920004f065c2571f32b7c28e7f850e9810 > xhci: Rename variables and reduce register reads. > > ba764f4b37e2bcce069f6c9d96b8374ee85257ed > USB 3.0 Hub Changes > > Loading usb-storage.ko module > Initializing USB Mass Storage driver... > usbcore: registered new interface driver usb-storage > USB Mass Storage support registered. > Waiting 8 seconds for driver initialization. > usb 7-1: New USB device found, idVendor=046d, idProduct=08af > usb 7-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0 > usb 7-2: new low speed USB device number 3 using uhci_hcd > usb 7-2: New USB device found, idVendor=0624, idProduct=0307 > usb 7-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0 > usb 7-2: Product: Avocent DSRIQ-USB > usb 7-2: Manufacturer: Avocent > input: Avocent Avocent DSRIQ-USB as > /devices/pci:00/:00:1d.2/usb7/7-2/7-2:1.0/input/input1 > generic-usb 0003:0624:0307.0001: input,hidraw0: USB HID v1.10 Keyboard > [Avocent Avocent DSRIQ-USB] on usb-:00:1d.2-2/input0 > input: Avocent Avocent DSRIQ-USB as > /devices/pci:00/:00:1d.2/usb7/7-2/7-2:1.1/input/input2 > generic-usb 0003:0624:0307.0002: input,hidraw1: USB HID v1.10 Mouse > [Avocent Avocent DSRIQ-USB] on usb-:00:1d.2-2/input1 > DRHD: handling fault status reg 602 > DMAR:[DMA Write] Request device [0b:00.1] fault addr f000 > DMAR:[fault reason 01] Present bit in root entry is clear > DRHD: handling fault status reg 702 > DMAR:[DMA Write] Request device [0b:00.0] fault addr f000 > DMAR:[fault reason 01] Present bit in root entry is clear > DRHD: handling fault status reg 2 > DMAR:[DMA Write] Request device [0b:00.1] fault addr f000 > DMAR:[fault reason 01] Present bit in root entry is clear > DRHD: handling fault status reg 102 > DMAR:[DMA Write] Request device [0b:00.0] fault addr f000 > DMAR:[fault reason 01] Present bit in root entry is clear > DRHD: handling fault status reg 202 > DMAR:[DMA Write] Request device [0b:00.1] fault addr f000 > DMAR:[fault reason 01] Present bit in root entry is clear > DRHD: handling fault status reg 302 > DMAR:[DMA Write] Request device [0b:00.0] fault addr f000 > DMAR:[fault reason 01] Present bit in root entry is clear > DRHD: handling fault status reg 402 > DMAR:[DMA Write] Request device [0b:00.0] fault addr ffbfb000 > DMAR:[fault reason 01] Present bit in root entry is clear > DRHD: handling fault status reg 502 > DMAR:[DMA Write] Request device [0b:00.1] fault addr f000 > DMAR:[fault reason 01] Present bit in root entry is clear > DRHD: handling fault status reg 602 > DMAR:[DMA Write] Request device [0b:00.0] fault addr f000 > DMAR:[fault reason 01] Present bit in root entry is clear > DRHD: handling fault status reg 702 > DMAR:[DMA Write] Request device [0b:00.1] fault addr f000 > DMAR:[fault reason 01] Present bit in root entry is clear > DRHD: handling fault status reg 2 > DMAR:[DMA Write] Request device [0b:00.0] fault addr f000 > DMAR:[fault reason 01] Present bit in root entry is clear > DRHD: handling fault status reg 102 > DMAR:[DMA Write] Request device [0b:00.1] fault addr f000 > DMAR:[fault reason 01] Present bit in root entry is clear > DRHD: handling fault st
kdump regression related to usb3 hub update
rite] Request device [0b:00.1] fault addr f000 DMAR:[fault reason 01] Present bit in root entry is clear DRHD: handling fault status reg 702 DMAR:[DMA Write] Request device [0b:00.0] fault addr f000 DMAR:[fault reason 01] Present bit in root entry is clear DRHD: handling fault status reg 2 DMAR:[DMA Write] Request device [0b:00.1] fault addr f000 DMAR:[fault reason 01] Present bit in root entry is clear DRHD: handling fault status reg 102 DMAR:[DMA Write] Request device [0b:00.0] fault addr f000 DMAR:[fault reason 01] Present bit in root entry is clear DRHD: handling fault status reg 202 DMAR:[DMA Write] Request device [0b:00.1] fault addr f000 DMAR:[fault reason 01] Present bit in root entry is clear DRHD: handling fault status reg 302 DMAR:[DMA Write] Request device [0b:00.0] fault addr f000 DMAR:[fault reason 01] Present bit in root entry is clear DRHD: handling fault status reg 402 DMAR:[DMA Write] Request device [0b:00.1] fault addr f000 DMAR:[fault reason 01] Present bit in root entry is clear DRHD: handling fault status reg 502 DMAR:[DMA Write] Request device [0b:00.0] fault addr f000 DMAR:[fault reason 01] Present bit in root entry is clear DRHD: handling fault status reg 602 DMAR:[DMA Write] Request device [0b:00.1] fault addr f000 DMAR:[fault reason 01] Present bit in root entry is clear DRHD: handling fault status reg 702 DMAR:[DMA Write] Request device [0b:00.0] fault addr f000 DMAR:[fault reason 01] Present bit in root entry is clear DRHD: handling fault status reg 2 DMAR:[DMA Write] Request device [0b:00.1] fault addr f000 DMAR:[fault reason 01] Present bit in root entry is clear DRHD: handling fault status reg 102 DMAR:[DMA Write] Request device [0b:00.0] fault addr f000 DMAR:[fault reason 01] Present bit in root entry is clear DRHD: handling fault status reg 202 DMAR:[DMA Write] Request device [0b:00.1] fault addr f000 DMAR:[fault reason 01] Present bit in root entry is clear DRHD: handling fault status reg 302 DMAR:[DMA Write] Request device [0b:00.0] fault addr f000 DMAR:[fault reason 01] Present bit in root entry is clear Initializing cgroup subsys cpuset Linux version 2.6.38-rc5+ .... Thanks. CAI Qian ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: kexec load failure introduced by "x86, memblock: Replace e820_/_early string with memblock_"
- "Yinghai Lu" wrote:
> Please check this one on top of tip or next.
This failed for both trees.
[r...@localhost linux-next]# patch -Np1
> Thanks
>
> Yinghai
>
> [PATCH] x86, memblock: Fix crashkernel allocation
>
> Cai Qian found that crashkernel is broken with x86 memblock changes
> 1. crashkernel=1...@32m always reported that range is used, even first
> kernel is small
>no one use that range
> 2. always get following report when using "kexec -p"
> Could not find a free area of memory of a000 bytes...
> locate_hole failed
>
> The root cause is that generic memblock_find_in_range() will try to
> get range from top_down.
> But crashkernel do need from low and specified range.
>
> Let's limit the target range with rash_base + crash_size to make sure
> that
> We get range from bottom.
>
> Reported-and-Bisected-by: CAI Qian
> Signed-off-by: Yinghai Lu
>
> ---
> arch/x86/kernel/setup.c | 19 ++-
> 1 file changed, 14 insertions(+), 5 deletions(-)
>
> Index: linux-2.6/arch/x86/kernel/setup.c
> ===
> --- linux-2.6.orig/arch/x86/kernel/setup.c
> +++ linux-2.6/arch/x86/kernel/setup.c
> @@ -516,19 +516,28 @@ static void __init reserve_crashkernel(v
>
> /* 0 means: find the address automatically */
> if (crash_base <= 0) {
> + unsigned long long start = 0;
> const unsigned long long alignment = 16<<20;/* 16M */
>
> - crash_base = memblock_find_in_range(alignment, ULONG_MAX,
> crash_size,
> - alignment);
> - if (crash_base == MEMBLOCK_ERROR) {
> + crash_base = alignment;
> + while (crash_base < 0x) {
> + start = memblock_find_in_range(crash_base,
> + crash_base + crash_size, crash_size, alignment);
> +
> + if (start == crash_base)
> + break;
> +
> + crash_base += alignment;
> + }
> + if (start != crash_base) {
> pr_info("crashkernel reservation failed - No suitable
> area
> found.\n");
> return;
> }
> } else {
> unsigned long long start;
>
> - start = memblock_find_in_range(crash_base, ULONG_MAX,
> crash_size,
> - 1<<20);
> + start = memblock_find_in_range(crash_base,
> + crash_base + crash_size, crash_size, 1<<20);
> if (start != crash_base) {
> pr_info("crashkernel reservation failed - memory is in
> use.\n");
> return;
>
> ___
> kexec mailing list
> kexec@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: kexec load failure introduced by "x86, memblock: Replace e820_/_early string with memblock_"
- "Yinghai Lu" wrote: > On 09/25/2010 11:55 PM, CAI Qian wrote: > >> > >> are you kexec from 2.6.35+ to 2.6.36-rc3+? > > No, both kernels were the same version. I am sorry the above logs > were misleading that were copy-and-pasted from different kernel > versions. > > can you check tip instead of next tree? I am wondering which patches there do you think would make the regression go away? > > Yinghai > > ___ > kexec mailing list > kexec@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/kexec ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: kexec load failure introduced by "x86, memblock: Replace e820_/_early string with memblock_"
- "Yinghai Lu" wrote: > On 09/25/2010 08:11 PM, caiq...@redhat.com wrote: > > # /sbin/kexec -p '--command-line=ro > root=/dev/mapper/VolGroup-lv_root rd_LVM_LV=VolGroup/lv_root > rd_LVM_LV=VolGroup/lv_swap rd_NO_LUKS rd_NO_MD rd_NO_DM > LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us > rhgb quiet console=tty0 console=ttyS0,115200 crashkernel=128M irqpoll > maxcpus=1 reset_devices cgroup_disable=memory ' > --initrd=/boot/initrd-2.6.36-rc3+kdump.img /boot/vmlinuz-2.6.36-rc3+ > > > > BUG: unable to handle kernel paging request at 8800dfffe400 > > IP: [] per_cpu_ptr_to_phys+0x3b/0x120 > > PGD 1a26063 PUD 1fffc067 PMD 1fffd067 PTE 0 > > Oops: [#1] SMP > > last sysfs file: /sys/devices/system/cpu/cpu0/crash_notes > > CPU 3 > > Modules linked in: ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 > iptable_filter ip_tables ip6t_REJECT nf_conntrack_ipv6 xt_state > nf_conntrack ip6table_filter ip6_tables ipv6 virtio_balloon pcspkr > 8139too 8139cp mii snd_intel8x0 snd_ac97_codec ac97_bus snd_seq > snd_seq_device snd_pcm snd_timer snd soundcore snd_page_alloc sg > i2c_piix4 i2c_core ext4 mbcache jbd2 floppy sd_mod crc_t10dif > virtio_pci virtio_ring virtio pata_acpi ata_generic ata_piix dm_mod > [last unloaded: scsi_wait_scan] > > > > Pid: 5671, comm: kexec Not tainted 2.6.35+ #11 /KVM > > RIP: 0010:[] [] > per_cpu_ptr_to_phys+0x3b/0x120 > > are you kexec from 2.6.35+ to 2.6.36-rc3+? No, both kernels were the same version. I am sorry the above logs were misleading that were copy-and-pasted from different kernel versions. > > Yinghai > > ___ > kexec mailing list > kexec@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/kexec ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: kvm kdump regression
- "Tejun Heo" wrote: > Hello, > > Thanks a lot for testing. Can you please verify whether the > following patch fixes the problem? Yes, it did. > > diff --git a/mm/percpu.c b/mm/percpu.c > index 58c572b..c76ef38 100644 > --- a/mm/percpu.c > +++ b/mm/percpu.c > @@ -1401,9 +1401,9 @@ int __init pcpu_setup_first_chunk(const struct > pcpu_alloc_info *ai, > > if (pcpu_first_unit_cpu == NR_CPUS) > pcpu_first_unit_cpu = cpu; > + pcpu_last_unit_cpu = cpu; > } > } > - pcpu_last_unit_cpu = cpu; > pcpu_nr_units = unit; > > for_each_possible_cpu(cpu) ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: kvm kdump regression
> Tejun, which version should I apply this patch for? Never mind. Fixed. Testing it right away... ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: kvm kdump regression
- "Tejun Heo" wrote:
> Hello,
>
> I was slightly confused with the debug patch but it looks like
> someone
> is feeding non-percpu address to the function. Hmm... or it could be
> that the first chunk test is returning false incorrectly for an
> address which is inside static percpu area. Can you please try the
> following patch instead? This should make the problem go away and
> give us enough information about which address is causing the
> problem.
> Please attach logs from both the original kernel before triggering
> crash and from the crash kernel.
Tejun, which version should I apply this patch for?
# patch -Np1
> Thank you.
>
> diff --git a/mm/percpu-vm.c b/mm/percpu-vm.c
> index 7d9c1d0..357569a 100644
> --- a/mm/percpu-vm.c
> +++ b/mm/percpu-vm.c
> @@ -415,6 +415,7 @@ static struct pcpu_chunk *pcpu_create_chunk(void)
> {
> struct pcpu_chunk *chunk;
> struct vm_struct **vms;
> + int i;
>
> chunk = pcpu_alloc_chunk();
> if (!chunk)
> @@ -429,6 +430,13 @@ static struct pcpu_chunk
> *pcpu_create_chunk(void)
>
> chunk->data = vms;
> chunk->base_addr = vms[0]->addr - pcpu_group_offsets[0];
> + chunk->chunkno = ++chunkno;
> + printk("XXX chunk %d allocated base_addr=%p\n",
> +chunk->chunkno, chunk->base_addr);
> + printk("XXX VMS:");
> + for (i = 0; i < pcpu_nr_groups; i++)
> + printk(" %...@%p", vms[i]->size, vms[i]->addr);
> + printk("\n");
> return chunk;
> }
>
> diff --git a/mm/percpu.c b/mm/percpu.c
> index 77e3f5a..14c836b 100644
> --- a/mm/percpu.c
> +++ b/mm/percpu.c
> @@ -96,7 +96,10 @@
> #define __pcpu_ptr_to_addr(ptr) (void __force *)(ptr)
> #endif /* CONFIG_SMP */
>
> +static int chunkno;
> +
> struct pcpu_chunk {
> + int chunkno;
> struct list_headlist; /* linked to pcpu_slot lists */
> int free_size; /* free bytes in the chunk */
> int contig_hint;/* max contiguous size hint */
> @@ -182,6 +185,40 @@ static struct list_head *pcpu_slot __read_mostly;
> /* chunk list slots */
> static void pcpu_reclaim(struct work_struct *work);
> static DECLARE_WORK(pcpu_reclaim_work, pcpu_reclaim);
>
> +void pcpu_dump_chunk(struct pcpu_chunk *chunk)
> +{
> + int i, contig = 0, free = 0;
> +
> + printk("XXX %d(f=%d,c=%d,u=%d,a=%d)", chunk->chunkno,
> +chunk->free_size, chunk->contig_hint,
> +chunk->map_used, chunk->map_alloc);
> + for (i = 0; i < chunk->map_used; i++) {
> + if (chunk->map[i] > 0) {
> + free += chunk->map[i];
> + contig = max(contig, chunk->map[i]);
> + }
> + printk(" %d", chunk->map[i]);
> + }
> + printk(" free=%d contig=%d%s\n", free, contig,
> +(free != chunk->free_size || contig != chunk->contig_hint) ?
> +" MISMATCH!" : "");
> +}
> +
> +void pcpu_dump_chunk_slots(void)
> +{
> + struct pcpu_chunk *chunk;
> + int i;
> +
> + printk("XXX percpu allocator dump\n");
> + for (i = 0; i < pcpu_nr_slots; i++) {
> + if (list_empty(&pcpu_slot[i]))
> + continue;
> + printk("XXX SLOT[%02d]\n", i);
> + list_for_each_entry(chunk, &pcpu_slot[i], list)
> + pcpu_dump_chunk(chunk);
> + }
> +}
> +
> static bool pcpu_addr_in_first_chunk(void *addr)
> {
> void *first_start = pcpu_first_chunk->base_addr;
> @@ -1017,8 +1054,20 @@ phys_addr_t per_cpu_ptr_to_phys(void *addr)
> return __pa(addr);
> else
> return page_to_phys(vmalloc_to_page(addr));
> - } else
> + } else {
> + if ((unsigned long)addr < VMALLOC_START ||
> + (unsigned long)addr >= VMALLOC_END) {
> + static bool first = true;
> + if (first) {
> + WARN_ON(1);
> + printk("XXX addr=%p out of vmalloc area and not
> in first
> chunk\n", addr);
> + pcpu_dump_chunk_slots();
> + first = false;
> + }
> + return __pa(addr);
> + }
> return page_to_phys(pcpu_addr_to_page(addr));
> + }
> }
>
> /**
> @@ -1283,6 +1332,11 @@ int __init pcpu_setup_first_chunk(const struct
> pcpu_alloc_info *ai,
> pcpu_chunk_struct_size = sizeof(struct pcpu_chunk) +
> BITS_TO_LONGS(pcpu_unit_pages) * sizeof(unsigned long);
>
> + printk("XXX CPU->UNIT M/O");
> + for_each_possible_cpu(cpu)
> + printk(" %d:%lx", unit_map[cpu], unit_off[cpu]);
> + printk("\n");
> +
> /*
>* Allocate chunk slots. The additional last slot is for
>* empty chunks.
> @@ -1292,6 +1346,10 @@ int __init pcpu_setup_first_chunk(const struct
> pcpu_alloc
Re: kvm kdump regression
- "Tejun Heo" wrote: > On 09/18/2010 02:36 PM, CAI Qian wrote: > >> Can you please apply the following patch, reproduce the problem > and > >> report the kernel log? > > Tejun, which version this patch is against? Both 2.6.36-rc2-mm1 and > 2.6.36-rc4-mm1 failed to apply it, > > The patch was against percpu#for-next branch. Below is a patch > regenerated against the current linux-next. Here were the messages when triggering kdump after applied this patch. SysRq : Trigger a crash BUG: unable to handle kernel NULL pointer dereference at (null) IP: [] sysrq_handle_crash+0x16/0x20 PGD c638be067 PUD c61eb1067 PMD 0 Oops: 0002 [#1] SMP last sysfs file: /sys/devices/virtio-pci/virtio2/block/vda/dev CPU 4 Modules linked in: ipv6 xfs exportfs virtio_net pcspkr snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd soundcore snd_page_alloc i2c_piix4 i2c_core ext4 mbcache jbd2 floppy virtio_blk virtio_pci virtio_ring virtio ata_generic pata_acpi ata_piix dm_mod [last unloaded: scsi_wait_scan] Pid: 5055, comm: bash Not tainted 2.6.36-rc4-mm1+ #24 /KVM RIP: 0010:[] [] sysrq_handle_crash+0x16/0x20 RSP: 0018:880c638a7e38 EFLAGS: 00010092 RAX: 0010 RBX: 0063 RCX: 16b5 RDX: RSI: 0046 RDI: 0063 RBP: 880c638a7e38 R08: R09: 8163f5a0 R10: 0001 R11: R12: 81a73480 R13: 0286 R14: 0007 R15: FS: 7f342adb9700() GS:88000288() knlGS: CS: 0010 DS: ES: CR0: 80050033 CR2: CR3: 000c63d8d000 CR4: 06e0 DR0: DR1: DR2: DR3: DR6: 0ff0 DR7: 0400 Process bash (pid: 5055, threadinfo 880c638a6000, task 880c608e7560) Stack: 880c638a7e78 812cc0b9 880c674bf680 0002 <0> 880c602e4d80 7f342adbb000 0002 fffb <0> 880c638a7ea8 812cc16a 880c0002 7f342adbb000 Call Trace: [] __handle_sysrq+0x129/0x190 [] write_sysrq_trigger+0x4a/0x50 [] proc_reg_write+0x76/0xb0 [] vfs_write+0xc8/0x190 [] sys_write+0x51/0x90 [] system_call_fastpath+0x16/0x1b Code: d0 88 81 43 6d 25 82 c9 c3 66 66 66 2e 0f 1f 84 00 00 00 00 00 55 48 89 e5 0f 1f 44 00 00 c7 05 99 ca b3 00 01 00 00 00 0f ae f8 04 25 00 00 00 00 01 c9 c3 55 48 89 e5 0f 1f 44 00 00 8d 47 RIP [] sysrq_handle_crash+0x16/0x20 RSP CR2: Initializing cgroup subsys cpuset Linux version 2.6.36-rc4-mm1+ (r...@localhost.localdomain) (gcc version 4.4.4 20100726 (Red Hat 4.4.4-13) (GCC) ) #24 SMP Mon Sep 20 03:44:24 EDT 2010 Command line: ro root=/dev/mapper/VolGroup-lv_root console=tty0 console=ttyS0,115200 SYSFONT=latarcyrheb-sun16 LANG=en_US.UTF-8 KEYTABLE=us no_console_suspend irqpoll maxcpus=1 reset_devices cgroup_disable=memory memmap=exactmap memmap=6...@0k memmap=1304...@49792k elfcorehdr=180204K memmap=64K$960K memmap=20K$3669996K memmap=272K$4194032K BIOS-provided physical RAM map: BIOS-e820: 0100 - 0009a800 (usable) BIOS-e820: 0009a800 - 000a (reserved) BIOS-e820: 000f - 0010 (reserved) BIOS-e820: 0010 - dfffb000 (usable) BIOS-e820: dfffb000 - e000 (reserved) BIOS-e820: fffbc000 - 0001 (reserved) BIOS-e820: 0001 - 000ca000 (usable) last_pfn = 0xca max_arch_pfn = 0x4 NX (Execute Disable) protection: active user-defined physical RAM map: user: - 000a (usable) user: 000f - 0010 (reserved) user: 030a - 0affb000 (usable) user: dfffb000 - e000 (reserved) user: fffbc000 - 0001 (reserved) DMI 2.4 present. No AGP bridge found last_pfn = 0xaffb max_arch_pfn = 0x4 PAT not supported by CPU. found SMP MP-table at [880f7d40] f7d40 init_memory_mapping: -0affb000 RAMDISK: 0ab4c000 - 0afef000 ACPI: RSDP 000f7cf0 00014 (v00 BOCHS ) ACPI: RSDT dfffd0d0 00030 (v01 BOCHS BXPCRSDT 0001 BXPC 0001) ACPI: FACP d4f0 00074 (v01 BOCHS BXPCFACP 0001 BXPC 0001) ACPI: DSDT dfffd630 01E4B (v01 BXPC BXDSDT 0001 INTL 20090123) ACPI: FACS d480 00040 ACPI: SSDT dfffd370 002B5 (v01 BOCHS BXPCSSDT 0001 BXPC 0001) ACPI: APIC dfffd100 001FA (v01 BOCHS BXPCAPIC 0001 BXPC 0001) No NUMA configuration found Faking a node at -0affb000 Initmem setup node 0 -0affb000 NODE_DATA [030a - 030c6fff] kvm-clock: Using msrs 12 and 11 kvm-clock: cpu 0, msr 0:4af5c01, boot cl
Re: kvm kdump regression
- "Tejun Heo" wrote:
> Hello,
>
> Can you please apply the following patch, reproduce the problem and
> report the kernel log?
Tejun, which version this patch is against? Both 2.6.36-rc2-mm1 and
2.6.36-rc4-mm1 failed to apply it,
patching file arch/x86/kernel/crash_dump_64.c
Hunk #2 FAILED at 36.
1 out of 2 hunks FAILED -- saving rejects to file
arch/x86/kernel/crash_dump_64.c.rej
patching file drivers/ata/libata-core.c
Hunk #1 succeeded at 6578 with fuzz 1 (offset 38 lines).
Hunk #2 FAILED at 6555.
1 out of 2 hunks FAILED -- saving rejects to file drivers/ata/libata-core.c.rej
patching file mm/percpu-vm.c
Hunk #1 FAILED at 415.
Hunk #2 FAILED at 429.
2 out of 2 hunks FAILED -- saving rejects to file mm/percpu-vm.c.rej
patching file mm/percpu.c
Hunk #1 FAILED at 90.
Hunk #2 succeeded at 176 with fuzz 1.
Hunk #3 succeeded at 1045 with fuzz 1.
Hunk #4 FAILED at 1473.
Hunk #5 FAILED at 1482.
Hunk #6 FAILED at 1501.
Hunk #7 FAILED at 1530.
Hunk #8 FAILED at 1639.
Hunk #9 FAILED at 1655.
7 out of 9 hunks FAILED -- saving rejects to file mm/percpu.c.rej
>
> Thanks.
>
> diff --git a/arch/x86/kernel/crash_dump_64.c
> b/arch/x86/kernel/crash_dump_64.c
> index 045b36c..8ac45a4 100644
> --- a/arch/x86/kernel/crash_dump_64.c
> +++ b/arch/x86/kernel/crash_dump_64.c
> @@ -13,6 +13,7 @@
> /* Stores the physical address of elf header of crash image. */
> unsigned long long elfcorehdr_addr = ELFCORE_ADDR_MAX;
>
> +void per_cpu_ptr_to_phys_failed(void *addr);
> /**
> * copy_oldmem_page - copy one page from "oldmem"
> * @pfn: page frame number to be copied
> @@ -35,8 +36,10 @@ ssize_t copy_oldmem_page(unsigned long pfn, char
> *buf,
> return 0;
>
> vaddr = ioremap(pfn << PAGE_SHIFT, PAGE_SIZE);
> - if (!vaddr)
> + if (!vaddr) {
> + per_cpu_ptr_to_phys_failed(vaddr);
> return -ENOMEM;
> + }
>
> if (userbuf) {
> if (copy_to_user(buf, vaddr + offset, csize)) {
> diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c
> index 932eaee..3bbffb0 100644
> --- a/drivers/ata/libata-core.c
> +++ b/drivers/ata/libata-core.c
> @@ -6540,6 +6540,8 @@ static void __init ata_parse_force_param(void)
> ata_force_tbl_size = idx;
> }
>
> +void per_cpu_ptr_to_phys_failed(void *addr);
> +
> static int __init ata_init(void)
> {
> int rc = -ENOMEM;
> @@ -6553,6 +6555,7 @@ static int __init ata_init(void)
> }
>
> printk(KERN_DEBUG "libata version " DRV_VERSION " loaded.\n");
> + per_cpu_ptr_to_phys_failed(0x1234);
> return 0;
> }
>
> diff --git a/mm/percpu-vm.c b/mm/percpu-vm.c
> index 7d9c1d0..357569a 100644
> --- a/mm/percpu-vm.c
> +++ b/mm/percpu-vm.c
> @@ -415,6 +415,7 @@ static struct pcpu_chunk *pcpu_create_chunk(void)
> {
> struct pcpu_chunk *chunk;
> struct vm_struct **vms;
> + int i;
>
> chunk = pcpu_alloc_chunk();
> if (!chunk)
> @@ -429,6 +430,13 @@ static struct pcpu_chunk
> *pcpu_create_chunk(void)
>
> chunk->data = vms;
> chunk->base_addr = vms[0]->addr - pcpu_group_offsets[0];
> + chunk->chunkno = ++chunkno;
> + printk("XXX chunk %d allocated base_addr=%p\n",
> +chunk->chunkno, chunk->base_addr);
> + printk("XXX VMS:");
> + for (i = 0; i < pcpu_nr_groups; i++)
> + printk(" %...@%p", vms[i]->size, vms[i]->addr);
> + printk("\n");
> return chunk;
> }
>
> diff --git a/mm/percpu.c b/mm/percpu.c
> index 58c572b..8ef744d 100644
> --- a/mm/percpu.c
> +++ b/mm/percpu.c
> @@ -90,7 +90,10 @@
>(unsigned long)__per_cpu_start)
> #endif
>
> +static int chunkno;
> +
> struct pcpu_chunk {
> + int chunkno;
> struct list_headlist; /* linked to pcpu_slot lists */
> int free_size; /* free bytes in the chunk */
> int contig_hint;/* max contiguous size hint */
> @@ -176,6 +179,40 @@ static struct list_head *pcpu_slot __read_mostly;
> /* chunk list slots */
> static void pcpu_reclaim(struct work_struct *work);
> static DECLARE_WORK(pcpu_reclaim_work, pcpu_reclaim);
>
> +void pcpu_dump_chunk(struct pcpu_chunk *chunk)
> +{
> + int i, contig = 0, free = 0;
> +
> + printk("XXX %d(f=%d,c=%d,u=%d,a=%d)", chunk->chunkno,
> +chunk->free_size, chunk->contig_hint,
> +chunk->map_used, chunk->map_alloc);
> + for (i = 0; i < chunk->map_used; i++) {
> + if (chunk->map[i] > 0) {
> + free += chunk->map[i];
> + contig = max(contig, chunk->map[i]);
> + }
> + printk(" %d", chunk->map[i]);
> + }
> + printk(" free=%d contig=%d%s\n", free, contig,
> +(free != chunk->free_size || contig != chunk->contig_hint) ?
> +" MISMATCH!" : "");
> +}
> +
> +void pcpu_dump_chunk_slots(void)
> +{
> + struct pcpu_chunk *chunk;
> + int i;
> +
> + printk("XXX perc
Re: kvm kdump regression
- "Tejun Heo" wrote: > On 09/14/2010 05:09 AM, CAI Qian wrote: > > This commit introduced a regression that an empty vmcore was > generated on a kvm guest. > > ioremap: invalid physical address db74 > > [ cut here ] > > WARNING: at arch/x86/mm/ioremap.c:83 __ioremap_caller+0x307/0x380() > > Hardware name: KVM > > Modules linked in: > > Pid: 1, comm: swapper Not tainted 2.6.36-rc2-mm1+ #2 > > Call Trace: > > [] warn_slowpath_common+0x7f/0xc0 > > [] ? copy_oldmem_page+0x4e/0xc0 > > [] warn_slowpath_null+0x1a/0x20 > > [] __ioremap_caller+0x307/0x380 > > [] ioremap_nocache+0x17/0x20 > > [] copy_oldmem_page+0x4e/0xc0 > > [] read_from_oldmem+0x7a/0xb0 > > [] merge_note_headers_elf64.clone.1+0x6c/0x21f > > [] ? copy_oldmem_page+0x8e/0xc0 > > [] ? read_from_oldmem+0x7a/0xb0 > > [] vmcore_init+0x19d/0x396 > > [] ? vmcore_init+0x0/0x396 > > [] do_one_initcall+0x43/0x190 > > [] kernel_init+0x2a0/0x330 > > [] kernel_thread_helper+0x4/0x10 > > [] ? kernel_init+0x0/0x330 > > [] ? kernel_thread_helper+0x0/0x10 > > ---[ end trace 93d72a36b9146f22 ]--- > > Kdump: vmcore not initialized > > > > commit 020ec6537aa65c18e9084c568d7b94727f2026fd > > Author: Tejun Heo > > Date: Fri Apr 9 18:57:00 2010 +0900 > > > > percpu: factor out pcpu_addr_in_first/reserved_chunk() and > update per_cpu_ptr_to_phys() > > Hmm... I'm confused. Isn't this the one fixed by commit 9983b6f0 > (percpu: fix first chunk match in per_cpu_ptr_to_phys())? No, it did not. 2.6.36-rc2-mm1 still had the same problem. > > Thanks. > > -- > tejun ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
kvm kdump regression
This commit introduced a regression that an empty vmcore was generated on a kvm
guest.
ioremap: invalid physical address db74
[ cut here ]
WARNING: at arch/x86/mm/ioremap.c:83 __ioremap_caller+0x307/0x380()
Hardware name: KVM
Modules linked in:
Pid: 1, comm: swapper Not tainted 2.6.36-rc2-mm1+ #2
Call Trace:
[] warn_slowpath_common+0x7f/0xc0
[] ? copy_oldmem_page+0x4e/0xc0
[] warn_slowpath_null+0x1a/0x20
[] __ioremap_caller+0x307/0x380
[] ioremap_nocache+0x17/0x20
[] copy_oldmem_page+0x4e/0xc0
[] read_from_oldmem+0x7a/0xb0
[] merge_note_headers_elf64.clone.1+0x6c/0x21f
[] ? copy_oldmem_page+0x8e/0xc0
[] ? read_from_oldmem+0x7a/0xb0
[] vmcore_init+0x19d/0x396
[] ? vmcore_init+0x0/0x396
[] do_one_initcall+0x43/0x190
[] kernel_init+0x2a0/0x330
[] kernel_thread_helper+0x4/0x10
[] ? kernel_init+0x0/0x330
[] ? kernel_thread_helper+0x0/0x10
---[ end trace 93d72a36b9146f22 ]---
Kdump: vmcore not initialized
commit 020ec6537aa65c18e9084c568d7b94727f2026fd
Author: Tejun Heo
Date: Fri Apr 9 18:57:00 2010 +0900
percpu: factor out pcpu_addr_in_first/reserved_chunk() and update
per_cpu_ptr_to_phys()
Factor out pcpu_addr_in_first/reserved_chunk() from
pcpu_chunk_addr_search() and use it to update per_cpu_ptr_to_phys()
such that it handles first chunk differently from the rest.
This patch doesn't cause any functional change and is to prepare for
percpu nommu support.
Signed-off-by: Tejun Heo
Reviewed-by: David Howells
Cc: Graff Yang
Cc: Sonic Zhang
diff --git a/mm/percpu.c b/mm/percpu.c
index 6e09741..1aeb081 100644
--- a/mm/percpu.c
+++ b/mm/percpu.c
@@ -177,6 +177,21 @@ static struct list_head *pcpu_slot __read_mostly; /* chunk
list slots */
static void pcpu_reclaim(struct work_struct *work);
static DECLARE_WORK(pcpu_reclaim_work, pcpu_reclaim);
+static bool pcpu_addr_in_first_chunk(void *addr)
+{
+ void *first_start = pcpu_first_chunk->base_addr;
+
+ return addr >= first_start && addr < first_start + pcpu_unit_size;
+}
+
+static bool pcpu_addr_in_reserved_chunk(void *addr)
+{
+ void *first_start = pcpu_first_chunk->base_addr;
+
+ return addr >= first_start &&
+ addr < first_start + pcpu_reserved_chunk_limit;
+}
+
static int __pcpu_size_to_slot(int size)
{
int highbit = fls(size);/* size is in bytes */
@@ -334,12 +349,10 @@ static void pcpu_chunk_relocate(struct pcpu_chunk *chunk,
int oslot)
*/
static struct pcpu_chunk *pcpu_chunk_addr_search(void *addr)
{
- void *first_start = pcpu_first_chunk->base_addr;
-
/* is it in the first chunk? */
- if (addr >= first_start && addr < first_start + pcpu_unit_size) {
+ if (pcpu_addr_in_first_chunk(addr)) {
/* is it in the reserved area? */
- if (addr < first_start + pcpu_reserved_chunk_limit)
+ if (pcpu_addr_in_reserved_chunk(addr))
return pcpu_reserved_chunk;
return pcpu_first_chunk;
}
@@ -1343,10 +1356,13 @@ bool is_kernel_percpu_address(unsigned long addr)
*/
phys_addr_t per_cpu_ptr_to_phys(void *addr)
{
- if ((unsigned long)addr < VMALLOC_START ||
- (unsigned long)addr >= VMALLOC_END)
- return __pa(addr);
- else
+ if (pcpu_addr_in_first_chunk(addr)) {
+ if ((unsigned long)addr < VMALLOC_START ||
+ (unsigned long)addr >= VMALLOC_END)
+ return __pa(addr);
+ else
+ return page_to_phys(vmalloc_to_page(addr));
+ } else
return page_to_phys(vmalloc_to_page(addr));
}
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [GIT PULL] Please pull x86 biarch crashdump support and misc devel
- "Eric W. Biederman" wrote: > ebied...@xmission.com (Eric W. Biederman) writes: > > > caiq...@redhat.com writes: > > > >>> I have fixed the wrong variable problem and I also spotted > another > >>> architecture test in crashdump-x86e that would only work on i386, > and > >>> fixed that. > >>> > >>> The new version is up at: > >>> > git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/kexec-tools-devel.git > >> # /sbin/kexec -p '--command-line=ro > root=/dev/mapper/vg_intels3e3601-lv_root > rd_LVM_LV=vg_intels3e3601/lv_root rd_LVM_LV=vg_intels3e3601/lv_swap > rd_NO_LUKS rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 > SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us > console=ttyS0,115200n81 irqpoll maxcpus=1 reset_devices > cgroup_disable=memory sysrq_always_enabled ' > --initrd=/boot/initrd-2.6.32-71.el6.bz623007.x86_64kdump.img > /boot/vmlinuz-2.6.32-71.el6.bz623007.x86_64 > >> Segmentation fault (core dumped) > > > > Thanks. Wow I am out of it. It appears that I was assuming > elf_info > > was zero initialized but taking no steps to make it that way, and > > someone elf_info.get_note_info took the value of one. > > > > I have added a memset and pushed that version of my code. > > I also took a moment and made certain that kexec -p at least works. > > > > I am surprised this bug didn't show up in my 32bit testing, *ugh*. > > > > So if you are brave and want to test there is a slim chance my new > > version of kexec will actually work on x86_64 now. > > Ok. This time I really have tested my new version of kexec on x86_64 > and it is working. At least enough to get me a crashdump that I > can read with vmcore-dmesg. So unless I think my horrible of trivial > mistakes is at an end. > > I would love to hear a test result from someone else, > and I apologize to everyone for messing up. It was also working for me, and the crash utility could open the vmcore and use several sub-commands without observing any problem. > > Eric > > ___ > kexec mailing list > kexec@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/kexec ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [GIT PULL] Please pull x86 biarch crashdump support and misc devel
> This failed to build for me. > > kexec/arch/i386/crashdump-x86.c: In function > ‘load_crashdump_segments’: > kexec/arch/i386/crashdump-x86.c:703: error: ‘KEXEC_ARCH_NATIVE’ > undeclared (first use in this function) > kexec/arch/i386/crashdump-x86.c:703: error: (Each undeclared > identifier is reported only once > kexec/arch/i386/crashdump-x86.c:703: error: for each function it > appears in.) > make: *** [kexec/arch/i386/crashdump-x86.o] Error 1 > > > > The active part of the fix is below (I have an earlier patch that > > just > > defines KEXEC_ARCH_NATIVE). I just realized you said the earlier patch defined KEXEC_ARCH_NATIVE. Now, kexec failed to load. /sbin/kexec -p '--command-line=ro root=/dev/mapper/vg_intels3e3601-lv_root rd_LVM_LV=vg_intels3e3601/lv_root rd_LVM_LV=vg_intels3e3601/lv_swap rd_NO_LUKS rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us console=ttyS0,115200n81 irqpoll maxcpus=1 reset_devices cgroup_disable=memory sysrq_always_enabled ' --initrd=/boot/initrd-2.6.36-rc2-mm1-fix+kdump.img /boot/vmlinuz-2.6.36-rc2-mm1-fix+ unsupported crashdump architecture: 5c42 Cannot load /boot/vmlinuz-2.6.36-rc2-mm1-fix+ > > > > Eric ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [GIT PULL] Please pull x86 biarch crashdump support and misc devel
- "Eric W. Biederman" wrote:
> caiq...@redhat.com writes:
>
> > BTW, kexec commandline was like this if helps.
> >
> > /sbin/kexec -p '--command-line=ro
> root=/dev/mapper/vg_intels3e3601-lv_root
> rd_LVM_LV=vg_intels3e3601/lv_root rd_LVM_LV=vg_intels3e3601/lv_swap
> rd_NO_LUKS rd_NO_MD rd_NO_DM LANG=en_US.UTF-8
> SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us
> console=ttyS0,115200n81 irqpoll maxcpus=1 reset_devices
> cgroup_disable=memory sysrq_always_enabled '
> --initrd=/boot/initrd-2.6.36-rc2-mm1-fix+kdump.img
> /boot/vmlinuz-2.6.36-rc2-mm1-fix+
> >
>
> Thanks for keeping me honest. It looks like I did something foolish,
> and wound up putting 386 headers on the x86_64 crash dump.
>
> I have rebased updated my kexec-tools-devel branch. Could you try it
> now? I haven't had a chance to test it yet but when I inspected the
> code it appeared very obvious what I had done wrong.
This failed to build for me.
kexec/arch/i386/crashdump-x86.c: In function ‘load_crashdump_segments’:
kexec/arch/i386/crashdump-x86.c:703: error: ‘KEXEC_ARCH_NATIVE’ undeclared
(first use in this function)
kexec/arch/i386/crashdump-x86.c:703: error: (Each undeclared identifier is
reported only once
kexec/arch/i386/crashdump-x86.c:703: error: for each function it appears in.)
make: *** [kexec/arch/i386/crashdump-x86.o] Error 1
>
> The active part of the fix is below (I have an earlier patch that
> just
> defines KEXEC_ARCH_NATIVE).
>
> Eric
> ---
>
> From 0a9bffb31c3aee53f58c62dde349c2842060c760 Mon Sep 17 00:00:00
> 2001
> From: Eric W. Biederman
> Date: Fri, 10 Sep 2010 11:57:36 -0700
> Subject: [PATCH] x86 crashdump: On x86_64 generate EM_X86_64 crashdump
> headers.
>
> Paper bag time. I forgot to double check my changes worked
> on x86_64. So I wound up with x86_64 putting i386 crashdump headers
> on when it generated a crashdump header. Ouch!
>
> Fix it by using the new KEXEC_ARCH_NATIVE value so we can see what
> we are doing.
>
> Signed-off-by: Eric W. Biederman
> ---
> kexec/arch/i386/crashdump-x86.c | 17 +++--
> 1 files changed, 15 insertions(+), 2 deletions(-)
>
> diff --git a/kexec/arch/i386/crashdump-x86.c
> b/kexec/arch/i386/crashdump-x86.c
> index 06e5ae9..e202e36 100644
> --- a/kexec/arch/i386/crashdump-x86.c
> +++ b/kexec/arch/i386/crashdump-x86.c
> @@ -690,19 +690,32 @@ int load_crashdump_segments(struct kexec_info
> *info, char* mod_cmdline,
> int nr_ranges, align = 1024, i;
> struct memory_range *mem_range, *memmap_p;
> struct crash_elf_info elf_info;
> + unsigned kexec_arch;
>
> /* Constant parts of the elf_info */
> elf_info.data = ELFDATA2LSB;
> elf_info.backup_src_start = BACKUP_SRC_START;
> elf_info.backup_src_end = BACKUP_SRC_END;
>
> + /* Get the architecture of the running kernel */
> + kexec_arch = elf_info.machine & KEXEC_ARCH_MASK;
> + if (kexec_arch == KEXEC_ARCH_DEFAULT)
> + kexec_arch = KEXEC_ARCH_NATIVE;
> +
> /* Get the elf architecture of the running kernel */
> - if ((info->kexec_flags & KEXEC_ARCH_MASK) == KEXEC_ARCH_X86_64) {
> + switch(kexec_arch) {
> + case KEXEC_ARCH_X86_64:
> elf_info.machine = EM_X86_64;
> - } else {
> + break;
> + case KEXEC_ARCH_386:
> elf_info.machine = EM_386;
> elf_info.lowmem_limit = X86_MAXMEM;
> elf_info.get_note_info = get_crash_notes;
> + break;
> + default:
> + fprintf(stderr, "unsupported crashdump architecture: %04x\n",
> + kexec_arch);
> + return -1;
> }
>
> if (get_crash_memory_ranges(&mem_range, &nr_ranges,
> --
> 1.6.5.2.143.g8cc62
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: fix /proc/vmcore seek
- "Arnd Bergmann" wrote:
> Commit 73296bc611 "procfs: Use generic_file_llseek in /proc/vmcore"
> broke
> seeking on /proc/vmcore. This changes it back to use default_llseek
> in order to restore the original behaviour.
>
> Signed-off-by: Arnd Bergmann
> Cc: Frederic Weisbecker
> Cc: sta...@kernel.org
>
> ---
> On Thursday 02 September 2010, CAI Qian wrote:
> >
> > Ken'ichi, it turned out this kernel patch caused makedumpfile
> failure below.
> > I am not sure if this is a kernel bug or makedumpfile needs to catch
> up with
> > the change.
>
> It doesn't matter who is correct here. A user program regressed
> because of a
> kernel change, so we need to fix the kernel. Thanks for the report!
>
> Please try this patch.
Confirmed makedumpfile is working again after applied this patch.
> --- a/fs/proc/vmcore.c
> +++ b/fs/proc/vmcore.c
> @@ -163,7 +163,7 @@ static ssize_t read_vmcore(struct file *file, char
> __user *buffer,
>
> static const struct file_operations proc_vmcore_operations = {
> .read = read_vmcore,
> - .llseek = generic_file_llseek,
> + .llseek = default_llseek,
> };
>
> static struct vmcore* __init get_new_element(void)
>
> ___
> kexec mailing list
> kexec@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: makedumpfile failed for 2.6.36-rc2
- "Ken'ichi Ohmichi" wrote: > Hi Cai, > > On Tue, 31 Aug 2010 04:29:02 -0400 (EDT) > CAI Qian wrote: > > > > > I'd like to reproduce the same problem on my machine, > > > > > could you please send your kernel .config file ? > > > > > > > > Attached. > > > > > > Thank you. > > > The linux-2.6.36-rc2 kernel is building now with your .config > file. > > > After building, I will test makedumpfile on the kernel. > > I built linux-2.6.36-rc2 kernel with your .config file, but it gets a > panic while 1st-kernel is booting. I'm not digging the cause. Another pointer is that 2.6.35-rc1 worked fine for me, while 2.6.35-rc4 and beyond started to fail. > So I built linux-2.6.36-rc2 kernel with my .config file instead, and > it > can boot and the kdump is successful. But makedumpfile runs > successfully, > so I cannot reproduce the problem which you reported. > > I will try linux-2.6.36-rc3 kernel with your .config file later. > BTW, do you have the way how I get the compressed copy of your > /proc/vmcore ? > I'd like to research the problem directly. > > > > > >> readmem: Can't seek the dump memory(/proc/vmcore). Invalid > > > argument > > > > > > lseek(2) failed with "Invalid argument", so the problem may be due > to > > > an > > > invalid offset. I'd like to know its value, so can you try the > > > attached > > > patch in the condition makedumpfile fails ? > > > And can you send the output of `readelf -a /proc/vmcore` ? > > Here you go, > > readmem: Can't seek the dump memory(/proc/vmcore). Invalid argument > > readmem: type_addr: 0, addr:880480027000, size:32 > > The above output does not include the offset value which should be > printed by my previous patch. Could you check the patch is applied > to right place (lseek(2) call in readmem() function) ? > In addiction, could you operate makedumpfile with "-D" option ? > > > Thanks > Ken'ichi Ohmichi > > ___ > kexec mailing list > kexec@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/kexec ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: kdump regression compared to v2.6.35
- "CAI Qian" wrote: > - "Tejun Heo" wrote: > > > Hello, > > > > On 08/30/2010 04:02 PM, CAI Qian wrote: > > >> Hmm, odd. So, here's the said debug patch. It will > periodically > > >> check all works and report if any work is being delayed for too > > long. > > >> If the max wait goes over 30secs, it will dump all task states > and > > >> disable itself. Can you please apply the patch on top of rc2 + > > >> wq#for-linus and report the output? It should tell us who's > stuck > > >> where. > > > > > > Nothing new was printed after around 10 minutes. > > > > Eh... that's interesting. That means no work is stalled on > > workqueues, so it at least is not a workqueue stall. Can you > please > > try the following then? Or if sysrq wasn't working because your > ... > SMP alternatives: switching to UP code > ACPI: Core revision 20100702 > ftrace: converting mcount calls to 0f 1f 44 00 00 > ftrace: allocating 18488 entries in 73 pages > XXX show_state_timer registered > Not enabling x2apic, Intr-remapping init failed. > Setting APIC routing to physical flat > ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 > CPU0: Intel(R) Xeon(R) CPU X7550 @ 2.00GHz stepping 06 > Brought up 1 CPUs > Total of 1 processors activated (3990.10 BogoMIPS). > devtmpfs: initialized > regulator: core version 0.5 > NET: Registered protocol family 16 > ACPI FADT declares the system doesn't support PCIe ASPM, so disable > it > ACPI: bus type pci registered > PCI: MMCONFIG for domain [bus 00-ff] at [mem > 0x8000-0x8fff] (base 0x8000) > PCI: MMCONFIG at [mem 0x8000-0x8fff] reserved in E820 > PCI: Using configuration type 1 for base access > bio: create slab at 0 Hit return too quickly, and here were something new, taskPC stack pid father swapper R running task0 1 0 0x 88046dc1bcf0 0046 81796812 81a90700 81c21420 00015d00 88106e5dc040 00015d00 88106e5dc5d0 88046dc1bfd8 88106e5dc5d8 88046dc1bfd8 Call Trace: [] __cond_resched+0x2a/0x40 [] _cond_resched+0x30/0x40 [] kmem_cache_alloc_node_notrace+0xa0/0x120 [] ? mempool_alloc_slab+0x0/0x20 [] mempool_create_node+0x41/0x1a0 [] ? mempool_free_slab+0x0/0x20 [] mempool_create+0x14/0x20 [] bioset_create+0x1a3/0x2f0 [] ? init_bio+0x0/0x141 [] init_bio+0xeb/0x141 [] ? default_bdi_init+0x9b/0xa2 [] do_one_initcall+0x43/0x190 [] kernel_init+0x2a0/0x330 [] kernel_thread_helper+0x4/0x10 [] ? kernel_init+0x0/0x330 [] ? kernel_thread_helper+0x0/0x10 kthreadd S 0 2 0 0x 88046dc1fe90 0046 88046dc1fe90 81013596 00015d00 88046dc1d4c0 00015d00 88046dc1da50 88046dc1ffd8 88046dc1da58 88046dc1ffd8 Call Trace: [] ? kernel_thread+0x76/0x80 [] ? kernel_thread_helper+0x0/0x10 [] kthreadd+0x275/0x280 [] kernel_thread_helper+0x4/0x10 [] ? kthreadd+0x0/0x280 [] ? kernel_thread_helper+0x0/0x10 ksoftirqd/0 S 0 3 2 0x 88046dc43ea0 0046 81a5a4a0 880002415d00 81a5a4a0 00015d00 88046dc1ca80 00015d00 88046dc1d010 88046dc43fd8 88046dc1d018 88046dc43fd8 Call Trace: [] run_ksoftirqd+0xe5/0x140 [] ? run_ksoftirqd+0x0/0x140 [] kthread+0x96/0xa0 [] kernel_thread_helper+0x4/0x10 [] ? kthread+0x0/0xa0 [] ? kernel_thread_helper+0x0/0x10 kworker/0:0 S 0 4 2 0x 88046dc45e50 0046 88046dc45db0 8104ac80 88046dc45e00 00015d00 88046dc1c040 00015d00 88046dc1c5d0 88046dc45fd8 88046dc1c5d8 88046dc45fd8 Call Trace: [] ? __dequeue_entity+0x30/0x50 [] worker_thread+0x259/0x3c0 [] ? worker_thread+0x0/0x3c0 [] kthread+0x96/0xa0 [] kernel_thread_helper+0x4/0x10 [] ? kthread+0x0/0xa0 [] ? kernel_thread_helper+0x0/0x10 kworker/u:0 S 0 5 2 0x 88046dc49e50 0046 88046dc49db0 8104ac80 88046dc49e00 00015d00 88046dc47500 00015d00 88046dc47a90 88046dc49fd8 88046dc47a98 88046dc49fd8 Call Trace: [] ? __dequeue_entity+0x30/0x50 [] worker_thread+0x259/0x3c0 [] ? worker_thread+0x0/0x3c0 [] kthread+0x96/0xa0 [] kernel_thread_helper+0x4/0x10 [] ? kthread+0x0/0xa0 [] ? kernel_thread_helper+0x0/0x10 migration/0 S 0 6 2 0x 88046dc4dde0 0046 88046dc4dd60 8104a7e8 88046dc4dd70 00015d00 88046dc46ac0 00015d00 880
Re: kdump regression compared to v2.6.35
- "Tejun Heo" wrote:
> Hello,
>
> On 08/30/2010 04:02 PM, CAI Qian wrote:
> >> Hmm, odd. So, here's the said debug patch. It will periodically
> >> check all works and report if any work is being delayed for too
> long.
> >> If the max wait goes over 30secs, it will dump all task states and
> >> disable itself. Can you please apply the patch on top of rc2 +
> >> wq#for-linus and report the output? It should tell us who's stuck
> >> where.
> >
> > Nothing new was printed after around 10 minutes.
>
> Eh... that's interesting. That means no work is stalled on
> workqueues, so it at least is not a workqueue stall. Can you please
> try the following then? Or if sysrq wasn't working because your
...
SMP alternatives: switching to UP code
ACPI: Core revision 20100702
ftrace: converting mcount calls to 0f 1f 44 00 00
ftrace: allocating 18488 entries in 73 pages
XXX show_state_timer registered
Not enabling x2apic, Intr-remapping init failed.
Setting APIC routing to physical flat
..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
CPU0: Intel(R) Xeon(R) CPU X7550 @ 2.00GHz stepping 06
Brought up 1 CPUs
Total of 1 processors activated (3990.10 BogoMIPS).
devtmpfs: initialized
regulator: core version 0.5
NET: Registered protocol family 16
ACPI FADT declares the system doesn't support PCIe ASPM, so disable it
ACPI: bus type pci registered
PCI: MMCONFIG for domain [bus 00-ff] at [mem 0x8000-0x8fff] (base
0x8000)
PCI: MMCONFIG at [mem 0x8000-0x8fff] reserved in E820
PCI: Using configuration type 1 for base access
bio: create slab at 0
> keyboard wasn't initialized at that point, you can setup serial
> console and trigger sysrq that way.
Yes, it was triggered from a serial console.
> Thanks.
>
> diff --git a/init/main.c b/init/main.c
> index 94ab488..e156b8f 100644
> --- a/init/main.c
> +++ b/init/main.c
> @@ -423,11 +423,19 @@ static void __init setup_command_line(char
> *command_line)
>
> static __initdata DECLARE_COMPLETION(kthreadd_done);
>
> +static void show_state_timer_fn(unsigned long data)
> +{
> + show_state();
> +}
> +static DEFINE_TIMER(show_state_timer, show_state_timer_fn, 0, 0);
> +
> static noinline void __init_refok rest_init(void)
> __releases(kernel_lock)
> {
> int pid;
>
> + printk("XXX show_state_timer registered\n");
> + mod_timer(&show_state_timer, jiffies + 180 * HZ);
> rcu_scheduler_starting();
> /*
>* We need to spawn init first so that it obtains pid 1, however
>
> ___
> kexec mailing list
> kexec@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: kdump regression compared to v2.6.35
- "Tejun Heo" wrote:
> On 08/30/2010 12:24 PM, CAI Qian wrote:
> > Can't see any difference with hangcheck timer enabled.
>
> Hmm, odd. So, here's the said debug patch. It will periodically
> check all works and report if any work is being delayed for too long.
> If the max wait goes over 30secs, it will dump all task states and
> disable itself. Can you please apply the patch on top of rc2 +
> wq#for-linus and report the output? It should tell us who's stuck
> where.
Nothing new was printed after around 10 minutes.
> Thanks.
>
> diff --git a/include/linux/workqueue.h b/include/linux/workqueue.h
> index f11100f..282322c 100644
> --- a/include/linux/workqueue.h
> +++ b/include/linux/workqueue.h
> @@ -83,6 +83,8 @@ struct work_struct {
> #ifdef CONFIG_LOCKDEP
> struct lockdep_map lockdep_map;
> #endif
> + unsigned long queued_on;
> + unsigned long activated_on;
> };
>
> #define WORK_DATA_INIT() ATOMIC_LONG_INIT(WORK_STRUCT_NO_CPU)
> diff --git a/kernel/workqueue.c b/kernel/workqueue.c
> index a2dccfc..9f95169 100644
> --- a/kernel/workqueue.c
> +++ b/kernel/workqueue.c
> @@ -913,6 +913,8 @@ static void insert_work(struct
> cpu_workqueue_struct *cwq,
> {
> struct global_cwq *gcwq = cwq->gcwq;
>
> + work->queued_on = work->activated_on = jiffies;
> +
> /* we own @work, set data and link */
> set_work_cwq(work, cwq, extra_flags);
>
> @@ -996,13 +998,14 @@ static void __queue_work(unsigned int cpu,
> struct workqueue_struct *wq,
> if (likely(cwq->nr_active < cwq->max_active)) {
> cwq->nr_active++;
> worklist = gcwq_determine_ins_pos(gcwq, cwq);
> + insert_work(cwq, work, worklist, work_flags);
> } else {
> work_flags |= WORK_STRUCT_DELAYED;
> worklist = &cwq->delayed_works;
> + insert_work(cwq, work, worklist, work_flags);
> + work->activated_on--;
> }
>
> - insert_work(cwq, work, worklist, work_flags);
> -
> spin_unlock_irqrestore(&gcwq->lock, flags);
> }
>
> @@ -1669,6 +1672,7 @@ static void cwq_activate_first_delayed(struct
> cpu_workqueue_struct *cwq)
> struct work_struct, entry);
> struct list_head *pos = gcwq_determine_ins_pos(cwq->gcwq, cwq);
>
> + work->activated_on = jiffies;
> move_linked_works(work, pos, NULL);
> __clear_bit(WORK_STRUCT_DELAYED_BIT, work_data_bits(work));
> cwq->nr_active++;
> @@ -2810,7 +2814,7 @@ struct workqueue_struct
> *__alloc_workqueue_key(const char *name,
>* list. Grab it, set max_active accordingly and add the new
>* workqueue to workqueues list.
>*/
> - spin_lock(&workqueue_lock);
> + spin_lock_irq(&workqueue_lock);
>
> if (workqueue_freezing && wq->flags & WQ_FREEZEABLE)
> for_each_cwq_cpu(cpu, wq)
> @@ -2818,7 +2822,7 @@ struct workqueue_struct
> *__alloc_workqueue_key(const char *name,
>
> list_add(&wq->list, &workqueues);
>
> - spin_unlock(&workqueue_lock);
> + spin_unlock_irq(&workqueue_lock);
>
> return wq;
> err:
> @@ -2849,9 +2853,9 @@ void destroy_workqueue(struct workqueue_struct
> *wq)
>* wq list is used to freeze wq, remove from list after
>* flushing is complete in case freeze races us.
>*/
> - spin_lock(&workqueue_lock);
> + spin_lock_irq(&workqueue_lock);
> list_del(&wq->list);
> - spin_unlock(&workqueue_lock);
> + spin_unlock_irq(&workqueue_lock);
>
> /* sanity check */
> for_each_cwq_cpu(cpu, wq) {
> @@ -2891,23 +2895,23 @@ void workqueue_set_max_active(struct
> workqueue_struct *wq, int max_active)
>
> max_active = wq_clamp_max_active(max_active, wq->flags, wq->name);
>
> - spin_lock(&workqueue_lock);
> + spin_lock_irq(&workqueue_lock);
>
> wq->saved_max_active = max_active;
>
> for_each_cwq_cpu(cpu, wq) {
> struct global_cwq *gcwq = get_gcwq(cpu);
>
> - spin_lock_irq(&gcwq->lock);
> + spin_lock(&gcwq->lock);
>
> if (!(wq->flags & WQ_FREEZEABLE) ||
> !(gcwq->flags & GCWQ_FREEZING))
> get_cwq(gcwq->cpu, wq)->max_active = max_active;
>
> - spin_unlock_irq(&gcwq->lock);
> + spin_unlock(&gcwq->lock);
> }
>
> - spin_unlock(&workqueue_lock);
&g
Re: kdump regression compared to v2.6.35
- "Tejun Heo" wrote: > Hello, > > On 08/30/2010 05:42 AM, CAI Qian wrote: > >> Hmmm... I can't reproduce it here. I wonder what the difference > is. > >> Can you please trigger sysrq-t after the boot is hung and post the > >> result? > > Sysrq keys did not work at this early stage of booting for me even > > adding sysrq_always_enabled. > > I see. I'll prepare a debug patch which tries to catch long > execution > delays in workqueue but in the meantime can you please turn on > hangcheck timer and see whether it triggers after the kernel hangs > during boot? Can't see any difference with hangcheck timer enabled. > Thanks. > > -- > tejun > > ___ > kexec mailing list > kexec@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/kexec ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: kdump regression compared to v2.6.35
- "Tejun Heo" wrote: > Hello, > > On 08/29/2010 02:03 PM, CAI Qian wrote: > > > > - "Tejun Heo" wrote: > > > >> On 08/29/2010 01:56 PM, CAI Qian wrote: > >>> It is easy to reproduce by passing maxcpus=1 to the first kernel. > >> > >> Do you mean booting w/ maxcpus=1 hangs the first kernel even w/o > >> kdump? > > Yes, here was the log, > > Hmmm... I can't reproduce it here. I wonder what the difference is. > Can you please trigger sysrq-t after the boot is hung and post the > result? Sysrq keys did not work at this early stage of booting for me even adding sysrq_always_enabled. > Thanks. > > -- > tejun ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: kdump regression compared to v2.6.35
- "Tejun Heo" wrote: > On 08/29/2010 01:56 PM, CAI Qian wrote: > > It is easy to reproduce by passing maxcpus=1 to the first kernel. > > Do you mean booting w/ maxcpus=1 hangs the first kernel even w/o > kdump? Yes, here was the log, Linux version 2.6.36-rc2-mm1-orig+ (r...@intel-s3e36-01.lab.bos.redhat.com) (gcc version 4.4.4 20100726 (Red Hat 4.4.4-13) (GCC) ) #3 SMP Sun Aug 29 07:27:06 EDT 2010 Command line: ro root=/dev/mapper/vg_intels3e3601-lv_root rd_LVM_LV=vg_intels3e3601/lv_root rd_LVM_LV=vg_intels3e3601/lv_swap rd_NO_LUKS rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us console=ttyS0,115200n81 crashkernel=128M maxcpus=1 BIOS-provided physical RAM map: BIOS-e820: - 00099800 (usable) BIOS-e820: 00099800 - 000a (reserved) BIOS-e820: 000e - 0010 (reserved) BIOS-e820: 0010 - 78c51000 (usable) BIOS-e820: 78c51000 - 78e66000 (ACPI NVS) BIOS-e820: 78e66000 - 79238000 (ACPI data) BIOS-e820: 79238000 - 792ac000 (reserved) BIOS-e820: 792ac000 - 792b9000 (ACPI data) BIOS-e820: 792b9000 - 792ce000 (reserved) BIOS-e820: 792ce000 - 792e1000 (ACPI data) BIOS-e820: 792e1000 - 792e3000 (reserved) BIOS-e820: 792e3000 - 792ec000 (ACPI data) BIOS-e820: 792ec000 - 792f8000 (reserved) BIOS-e820: 792f8000 - 792f9000 (ACPI data) BIOS-e820: 792f9000 - 792ff000 (reserved) BIOS-e820: 792ff000 - 7932 (ACPI data) BIOS-e820: 7932 - 79341000 (reserved) BIOS-e820: 79341000 - 7937 (ACPI data) BIOS-e820: 7937 - 793b1000 (reserved) BIOS-e820: 793b1000 - 7996a000 (ACPI data) BIOS-e820: 7996a000 - 79b6a000 (ACPI NVS) BIOS-e820: 79b6a000 - 79cae000 (ACPI data) BIOS-e820: 79cae000 - 79cde000 (reserved) BIOS-e820: 79cde000 - 79d72000 (ACPI data) BIOS-e820: 79d72000 - 79d75000 (reserved) BIOS-e820: 79d75000 - 79e05000 (ACPI data) BIOS-e820: 79e05000 - 79e7 (reserved) BIOS-e820: 79e7 - 7bd5f000 (ACPI data) BIOS-e820: 7bd5f000 - 7be4f000 (reserved) BIOS-e820: 7be4f000 - 7bf87000 (ACPI data) BIOS-e820: 7bf87000 - 7bfcf000 (ACPI NVS) BIOS-e820: 7bfcf000 - 7bfff000 (ACPI data) BIOS-e820: 7bfff000 - 9000 (reserved) BIOS-e820: fc00 - fd00 (reserved) BIOS-e820: fed1c000 - fed2 (reserved) BIOS-e820: ff00 - 0001 (reserved) BIOS-e820: 0001 - 00108000 (usable) NX (Execute Disable) protection: active DMI 2.5 present. No AGP bridge found last_pfn = 0x108 max_arch_pfn = 0x4 x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106 last_pfn = 0x78c51 max_arch_pfn = 0x4 found SMP MP-table at [880fdd80] fdd80 init_memory_mapping: -78c51000 init_memory_mapping: 0001-00108000 RAMDISK: 3754d000 - 37ff Reserving 128MB of memory at 48MB for crashkernel (System RAM: 67584MB) ACPI: RSDP 000f0410 00024 (v02 QUANTA) ACPI: XSDT 7bffe120 000B4 (v01 QUANTA QSSC-S4R 0113) ACPI: FACP 7bffd000 000F4 (v04 QUANTA QSSC-S4R MSFT 010D) ACPI: DSDT 7bfe3000 19F6A (v02 QUANTA QSSC-S4R 0003 MSFT 010D) ACPI: FACS 7bf87000 00040 ACPI: APIC 7bfe2000 003E4 (v02 QUANTA QSSC-S4R MSFT 010D) ACPI: MSCT 7bfe1000 00090 (v01 QUANTA QSSC-S4R 0001 MSFT 010D) ACPI: MCFG 7bfe 0003C (v01 QUANTA QSSC-S4R 0001 MSFT 010D) ACPI: HPET 7bfdf000 00038 (v01 QUANTA QSSC-S4R 0001 MSFT 010D) ACPI: SLIT 7bfde000 0003C (v01 QUANTA QSSC-S4R 0001 MSFT 010D) ACPI: SRAT 7bfdd000 00930 (v02 QUANTA QSSC-S4R 0001 MSFT 010D) ACPI: SPCR 7bfdc000 00050 (v01 QUANTA QSSC-S4R MSFT 010D) ACPI: WDDT 7bfdb000 00040 (v01 QUANTA QSSC-S4R MSFT 010D) ACPI: SSDT 7bf4a000 3CFA4 (v02 QUANTA QSSC-S4R 4000 INTL 20061109) ACPI: SSDT 7bfda000 00174 (v02 QUANTA QSSC-S4R 4000 INTL 20061109) ACPI: PMCT 7bfd9000 00060 (v01 QUANTA QSSC-S4R MSFT 010D) ACPI: MIGT 7bfd8000 00040 (v01 QUANTA QSSC-S4R MSFT 010D) ACPI: TCPA 7bfd5000 00032 (v00 QUANTA QSSC-S4R ) ACPI: HEST 7bfd4000 000A8 (v01 QUANTA QSSC-S4R 0001 INTL 0001) ACPI: BERT 7bfd3000 00030 (v01 QUANTA QSSC-S4R 0001 INTL 0001) ACPI: ERST 7bfd2000 00230 (v01 QUANTA QSSC-S4R
Re: kdump regression compared to v2.6.35
- caiq...@redhat.com wrote: > - "Tejun Heo" wrote: > > > Hello, > > > > On 08/29/2010 01:24 PM, CAI Qian wrote: > > >> On 08/29/2010 09:01 AM, caiq...@redhat.com wrote: > > >>> Further bisect indicated this bad commit from the merge. Given > kdump > > >>> kernel was running with maxcpus=1, I guess this work caused > > fs/bio.c > > >>> hung in the workqueue on UP. Reverted the whole merge let kdump > > work > > >>> again. > > >> > > >> Can you please pull from the following git tree and see whether > it > > >> fixes the problem? There was a bug in nr_active accounting. > > > > > > It had the same problem. > > > > I see. Hmm... a different issue then. Can you please tell me how > to > > reproduce the problem? It is easy to reproduce by passing maxcpus=1 to the first kernel. > First, to configure kdump - see Documentation/kdump/kdump.txt. It > might be easier if you are using a distro that provide advanced kdump > tools. Then, to trigger the kdump - echo c >/proc/sysrq-trigger. In > case needed, here is the system information, > # lscpu > Architecture: x86_64 > CPU op-mode(s):32-bit, 64-bit > CPU(s):64 > Thread(s) per core:2 > Core(s) per socket:8 > CPU socket(s): 4 > NUMA node(s): 4 > Vendor ID: GenuineIntel > CPU family:6 > Model: 46 > Stepping: 6 > CPU MHz: 1064.000 > Virtualization:VT-x > L1d cache: 32K > L1i cache: 32K > L2 cache: 256K > L3 cache: 18432K > NUMA node0 CPU(s): 0,4,8,12,16,20,24,28,32,36,40,44,48,52,56,60 > NUMA node1 CPU(s): 1,5,9,13,17,21,25,29,33,37,41,45,49,53,57,61 > NUMA node2 CPU(s): 2,6,10,14,18,22,26,30,34,38,42,46,50,54,58,62 > NUMA node3 CPU(s): 3,7,11,15,19,23,27,31,35,39,43,47,51,55,59,63 > > > Thanks. > > > > -- > > tejun > > > > ___ > > kexec mailing list > > kexec@lists.infradead.org > > http://lists.infradead.org/mailman/listinfo/kexec ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: kdump regression compared to v2.6.35
- "Tejun Heo" wrote: > On 08/29/2010 09:01 AM, caiq...@redhat.com wrote: > > Further bisect indicated this bad commit from the merge. Given kdump > > kernel was running with maxcpus=1, I guess this work caused fs/bio.c > > hung in the workqueue on UP. Reverted the whole merge let kdump work > > again. > > Can you please pull from the following git tree and see whether it > fixes the problem? There was a bug in nr_active accounting. It had the same problem. > git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq.git for-linus > > Thanks. > > -- > tejun > > ___ > kexec mailing list > kexec@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/kexec ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
kdump regression compared to v2.6.35
Just a head-up, the kdump kernel is stuck here. Bisect indicated that cc41f5cede3c63836d1c0958204630b07f5b5ee7 was also good. Kernel command line: ro root=/dev/mapper/vg_intels3e3601-lv_root rd_LVM_LV=vg_intels3e3601/lv_root rd_LVM_LV=vg_intels3e3601/lv_swap rd_NO_LUKS rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us console=ttyS0,115200 irqpoll maxcpus=1 reset_devices cgroup_disable=memory memmap=exactmap memmap=6...@0k memmap=1304...@49792k elfcorehdr=180204K memmap=128K$896K memmap=2124K#1978520K memmap=4280K#1980644K memmap=464K$1984924K memmap=156K#1985388K memmap=84K$1985544K memmap=92K#1985628K memmap=8K$1985720K memmap=124K#1985728K memmap=136K$1985852K memmap=192K#1985988K memmap=260K$1986180K memmap=2816K#1986440K memmap=176K$1989256K memmap=3028K#1989432K memmap=2048K#1992460K memmap=2072K#1994508K memmap=8K$1996580K memmap=324K#1996588K memmap=428K$1996912K memmap=31584K#1997340K memmap=960K$2028924K memmap=1248K#2029884K memmap=288K#2031132K memmap=192K#2031420K memmap=327684K$2031612K memmap=16384K$4128768K memmap=16K$4174960K memmap=16384K$417! 7920K Misrouted IRQ fixup and polling support enabled This may significantly impact system performance Disabling memory control group subsystem PID hash table entries: 512 (order: 0, 4096 bytes) Checking aperture... No AGP bridge found Queued invalidation will be enabled to support x2apic and Intr-remapping. Subtract (141 early reservations) #1 [000400 - 00052b1548] TEXT DATA BSS #2 [000ab63000 - 000afef000] RAMDISK #3 [00052b2000 - 00052b2220] BRK #4 [09b000 - 0fdd20] BIOS reserved #5 [0fdd30 - 10] BIOS reserved #6 [0fdd20 - 0fdd30]MP-table mpf #7 [12 - 00f012]MP-table mpc #8 [01 - 012000] TRAMPOLINE #9 [012000 - 016000] ACPI WAKEUP #10 [016000 - 017000] PGTABLE #11 [017000 - 01703c] ACPI SLIT #12 [00f040 - 00f4c0] MEMNODEMAP #13 [00030a - 00030c7000] NODE_DATA #14 [00030c7000 - 00030c8000] BOOTMEM #15 [00034c8000 - 00034c8030] BOOTMEM #16 [00038c9000 - 00038ca000] BOOTMEM #17 [00038ca000 - 00038cb000] BOOTMEM #18 [0003a0 - 0003e0]MEMMAP 0 #19 [00030c8000 - 00030e] BOOTMEM #20 [00030e - 00030f8000] BOOTMEM #21 [00030f8000 - 00030f9000] BOOTMEM #22 [00030f9000 - 00030f9041] BOOTMEM #23 [00030f9080 - 00030f9149] BOOTMEM #24 [00030f9180 - 00030f9768] BOOTMEM #25 [00030f9780 - 00030f97e8] BOOTMEM #26 [00030f9800 - 00030f9868] BOOTMEM #27 [00030f9880 - 00030f98e8] BOOTMEM #28 [00030f9900 - 00030f9968] BOOTMEM #29 [00030f9980 - 00030f99e8] BOOTMEM #30 [00030f9a00 - 00030f9a68] BOOTMEM #31 [00030f9a80 - 00030f9ae8] BOOTMEM #32 [00030f9b00 - 00030f9b68] BOOTMEM #33 [00030f9b80 - 00030f9be8] BOOTMEM #34 [00030f9c00 - 00030f9c68] BOOTMEM #35 [00030f9c80 - 00030f9ce8] BOOTMEM #36 [00030f9d00 - 00030f9d68] BOOTMEM #37 [00030f9d80 - 00030f9de8] BOOTMEM #38 [00030f9e00 - 00030f9e68] BOOTMEM #39 [00030f9e80 - 00030f9ee8] BOOTMEM #40 [00030f9f00 - 00030f9f68] BOOTMEM #41 [00030f9f80 - 00030f9fe8] BOOTMEM #42 [00030fa000 - 00030fa068] BOOTMEM #43 [00030fa080 - 00030fa0e8] BOOTMEM #44 [00030fa100 - 00030fa168] BOOTMEM #45 [00030fa180 - 00030fa1e8] BOOTMEM #46 [00030fa200 - 00030fa268] BOOTMEM #47 [00030fa280 - 00030fa2e8] BOOTMEM #48 [00030fa300 - 00030fa368] BOOTMEM #49 [00030fa380 - 00030fa3e8] BOOTMEM #50 [00030fa400 - 00030fa468] BOOTMEM #51 [00030fa480 - 00030fa4e8] BOOTMEM #52 [00030fa500 - 00030fa568] BOOTMEM #53 [00030fa580 - 00030fa5e8] BOOTMEM #54 [00030fa600 - 00030fa668] BOOTMEM #55 [00030fa680 - 00030fa6e8] BOOTMEM #56 [00030fa700 - 00030fa768] BOOTMEM #57 [00030fa780 - 00030fa7e8] BOOTMEM #58 [00030fa800 - 00030fa820] BOOTMEM #59 [00030fa840 - 00030fac0e] BOOTMEM #60 [00030fac40 - 00030fb00e] BOOTMEM #61 [000540 - 000541e000] BOOTMEM #62 [000542 - 000543e000] BOOTMEM #63 [000544 - 000545e000] BOOTMEM #64 [000546 - 000547e000] BOOTMEM #65 [000548 - 000549e000] BOOTMEM #66 [00054a - 00054be000] BOOTMEM #67 [00054c - 00054de000] BOOTMEM #68 [00054e - 00054fe000] BOOTMEM #69 [000550 - 000551e000] BOOTMEM #70 [000552 - 000553e000] BOOTMEM #71 [000554 - 000555e000] BOOTMEM #72 [000556 - 000557e000] BOOTMEM #73 [000558 - 000559e000] BOOTMEM #74 [00055a - 00055b
Re: [PATCH 0/5] make '-x'/'-i' option unnecessary for re-filtering.
Hello Ken'ichi! This series of patches is working great here. Thanks! CAI Qian From: "Ken'ichi Ohmichi" Subject: [PATCH 0/5] make '-x'/'-i' option unnecessary for re-filtering. Date: Tue, 13 Oct 2009 14:18:44 +0900 > > Hi, > > Ken'ichi Ohmichi wrote: >> Hi CAI, >> >> Thank you for testing and a good point. >> >> CAI Qian wrote: >>>> makedumpfile version 1.3.4 is released. >>>> Your comments/patches are welcome. >>>> >>>> Changelog: >>>> o New feature >>>>- Support kdump-compressed format for input file. (by Takao Indoh) >>>> A new makedumpfile can read dumpfile which is kdump-compressed format >>>> instead of /proc/vmcore and re-filter it. >>> Does it need to specify either a vmlinux or vmcoreinfo file? >> >> Yes, we need to do it for re-filtering now. >> But it is not difficult to implement a feature that it is not >> necessary to specify either a vmlinux or vmcoreinfo file for >> re-filtering. I will try to implement the feature. > > I created the patchset for implementing the above feature. > Any comments/patches are welcome. > > --- > [PATCH 0/5] make '-x'/'-i' option unnecessary for re-filtering. > > makedumpfile v1.3.4 can read a dumpfile which is kdump-compressed > format instead of /proc/vmcore and re-filter it. > > When re-filtering, it is necessary to specify '-x'/'-i' option. > If not specifying, makedumpfile fails like the following: > > # ./makedumpfile -c -d 3 vmcore vmcore.1 > ... > The dumpfile is saved to vmcore.1 > ... > # > # /makedumpfile -c -d 31 vmcore.1 vmcore.2 > vmcore.1 doesn't contain vmcoreinfo. > Specify '-x' option or '-i' option. > Commandline parameter is invalid. > Try `makedumpfile --help' for more information. > # > > For usability, this patchset makes '-x'/'-i' option unnecessary > for re-filtering. > > > Thanks > Ken'ichi Ohmichi > > > ___ > kexec mailing list > kexec@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/kexec ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: makedumpfile-1.3.4: Support kdump-compressed format for input file.
Hallo Ken'ichi! From: "Ken'ichi Ohmichi" Subject: makedumpfile-1.3.4: Support kdump-compressed format for input file. Date: Fri, 09 Oct 2009 11:40:35 +0900 > > Hi, > > makedumpfile version 1.3.4 is released. > Your comments/patches are welcome. > > Changelog: > o New feature >- Support kdump-compressed format for input file. (by Takao Indoh) > A new makedumpfile can read dumpfile which is kdump-compressed format > instead of /proc/vmcore and re-filter it. Does it need to specify either a vmlinux or vmcoreinfo file? Got a failure if not. Note, file vmcore is a full VMCore generated from a 2.6.31 x86_64 kernel and kexec-tools-2.0.0. # ./makedumpfile -c -d 3 vmcore vmcore.1 ... The dumpfile is saved to vmcore.1 ... # /makedumpfile -c -d 31 vmcore.1 vmcore.2 vmcore.1 doesn't contain vmcoreinfo. Specify '-x' option or '-i' option. Commandline parameter is invalid. Try `makedumpfile --help' for more information. makedumpfile Failed. Thanks! CAI Qian >- Add dump filtering on an x86_64 xen domain-0. > A new makedumpfile can exclude unnecessary pages (cache pages, user > process data pages, and free pages) on on x86_64 xen domain-0. >- Save utsname data into disk_dump_header. > A new makedumpfile saves utsname data into disk_dump_header. > The crash utility can output the data from a dumpfile by 'help -n'. >- Add linux-2.6.30 - 2.6.31 support. > > o Bugfix >- Fix some lacks of free(). (by Minoru Usui) > > o Code Cleanup >- Add some comments to code for the readability. >- Remove VERSION_LINUX_2_6_XX. >- Add paddr_to_pfn() and pfn_to_paddr() macros. >- Merge the version information of makedumpfile. >- Merge the PT_NOTE methods of both elf64 and elf32. >- Reduce some indents in exclude_unnecessary_pages(). >- Remove unnecessary spaces. >- read_disk_dump_header() and read_kdump_sub_header(). >- Use DISKDUMP_HDADER_BLOCKS for the readability. > > Explanation of makedumpfile: > To shorten the size of the dumpfile and the time of creating the > dumpfile, makedumpfile copies only the necessary pages for analysis > to the dumpfile from /proc/vmcore. You can specify the kind of > unnecessary pages with dump_level. If you want to shorten the size > further, enable the compression of the page data. > > Download: > You can download the latest makedumpfile from the following URL. > Details of the change are written on the CVS page of the following site. > https://sourceforge.net/projects/makedumpfile/ > > Method of installation: > You can compile the makedumpfile command as follows; > 1. "tar -zxvf makedumpfile-x.y.z.tar.gz" > 2. "cd makedumpfile-x.y.z" > 3. "make; make install" > > Usage: > makedumpfile [-c] [-E] [-d dump_level] [-x vmlinux] dump_mem dump_file > > Example: > If you want to exclude pages filled by zero, cache pages, user pages > and free pages and to enable compression, please execute the following > command. > > # makedumpfile -c -d 31 -x vmlinux /proc/vmcore dumpfile > > > Thanks > Ken'ichi Ohmichi > ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: Kdump Failed with 2.6.31 i386
From: Amerigo Wang Subject: Re: Kdump Failed with 2.6.31 i386 Date: Fri, 09 Oct 2009 15:44:53 +0800 > CAI Qian wrote: >> Hallo! >> Is it a known issue that kdump failed with 2.6.31 i386 systems? The >> kdump kernel seems hang here. >> ... >> Spurious LAPIC timer interrupt on cpu 0 >> Console: colour VGA+ 80x25 >> console handover: boot [earlyser0] -> real [ttyS0] >> Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar >> ... MAX_LOCKDEP_SUBCLASSES: 8 >> ... MAX_LOCK_DEPTH: 48 >> ... MAX_LOCKDEP_KEYS:8191 >> ... CLASSHASH_SIZE: 4096 >> ... MAX_LOCKDEP_ENTRIES: 16384 >> ... MAX_LOCKDEP_CHAINS: 32768 >> ... CHAINHASH_SIZE: 16384 >> memory used by lock dependency info: 3743 kB >> per task-struct memory footprint: 1920 bytes >> HPET: 3 timers in total, 0 timers will be used for per-cpu timer >> Clocksource tsc unstable (delta = 577041676100 ns) >> > > > This case hangs on setup_irq(0, &irq0). We already knew this. This has been seen on 3 HP and 1 DELL servers. > >> or >> Spurious LAPIC timer interrupt on cpu 0 >> Console: colour VGA+ 80x25 >> console [ttyS0] enabled >> Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar >> ... MAX_LOCKDEP_SUBCLASSES: 8 >> ... MAX_LOCK_DEPTH: 48 >> ... MAX_LOCKDEP_KEYS:8191 >> ... CLASSHASH_SIZE: 4096 >> ... MAX_LOCKDEP_ENTRIES: 16384 >> ... MAX_LOCKDEP_CHAINS: 32768 >> ... CHAINHASH_SIZE: 16384 >> memory used by lock dependency info: 3743 kB >> per task-struct memory footprint: 1920 bytes >> allocated 2621440 bytes of page_cgroup >> please try 'cgroup_disable=memory' option if you don't want memory >> cgroups >> Calibrating delay loop (skipped), value calculated using timer >> frequency.. 5986.01 BogoMIPS (lpj=2993008) > > This case hangs after calibrate_delay() which already passed > late_time_init(), and probably before security_init()... > (since probably you have CONFIG_SECURITY=y) > This has been observed on a NEC server. Thanks! CAI Qian > Did you meet these two cases on the same machine? Or on different > machines? > > Thanks. > > ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Kdump Failed with 2.6.31 i386
Hallo! Is it a known issue that kdump failed with 2.6.31 i386 systems? The kdump kernel seems hang here. ... Spurious LAPIC timer interrupt on cpu 0 Console: colour VGA+ 80x25 console handover: boot [earlyser0] -> real [ttyS0] Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar ... MAX_LOCKDEP_SUBCLASSES: 8 ... MAX_LOCK_DEPTH: 48 ... MAX_LOCKDEP_KEYS:8191 ... CLASSHASH_SIZE: 4096 ... MAX_LOCKDEP_ENTRIES: 16384 ... MAX_LOCKDEP_CHAINS: 32768 ... CHAINHASH_SIZE: 16384 memory used by lock dependency info: 3743 kB per task-struct memory footprint: 1920 bytes HPET: 3 timers in total, 0 timers will be used for per-cpu timer Clocksource tsc unstable (delta = 577041676100 ns) or Spurious LAPIC timer interrupt on cpu 0 Console: colour VGA+ 80x25 console [ttyS0] enabled Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar ... MAX_LOCKDEP_SUBCLASSES: 8 ... MAX_LOCK_DEPTH: 48 ... MAX_LOCKDEP_KEYS:8191 ... CLASSHASH_SIZE: 4096 ... MAX_LOCKDEP_ENTRIES: 16384 ... MAX_LOCKDEP_CHAINS: 32768 ... CHAINHASH_SIZE: 16384 memory used by lock dependency info: 3743 kB per task-struct memory footprint: 1920 bytes allocated 2621440 bytes of page_cgroup please try 'cgroup_disable=memory' option if you don't want memory cgroups Calibrating delay loop (skipped), value calculated using timer frequency.. 5986.01 BogoMIPS (lpj=2993008) Thanks! CAI Qian ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: Can't exclude unnecessary pages for 2.6.31 Kernel
From: Dave Anderson Subject: Re: Can't exclude unnecessary pages for 2.6.31 Kernel Date: Fri, 2 Oct 2009 10:24:05 -0400 (EDT) > > - "Dave Anderson" wrote: > >> - "CAI Qian" wrote: >> >> > From: CAI Qian >> > Subject: Re: Can't exclude unnecessary pages for 2.6.31 Kernel >> > Date: Fri, 02 Oct 2009 19:26:44 +0800 (CST) >> > >> > > Cced the crash utility maintainer. >> > > >> > > BZ filed -- https://bugzilla.redhat.com/show_bug.cgi?id=526749 >> > > >> > >> It does make -d 16 works without errors. However, the crash utility >> > >> failed to read it (the log attached), but the full VMCore is working >> > >> fine there. >> > >> >> > >> crash: page excluded: kernel virtual address: 819f1764 type: >> > >> "tss_struct ist array" >> > >> >> I believe that the "tss_struct" issue is a crash bug if in fact it works >> OK with a full vmcore. I'll look at the vmlinux/vmcores from the BZ. > > Definitely a crash utility bug -- I'll fix it in the next upstream release. > > Dave > After applied the patch, the problem has gone, and I can analyze the VMCore without any problem so far. Thanks Ken'ichi and Dave for the fixes! CAI Qian ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: Can't exclude unnecessary pages for 2.6.31 Kernel
From: CAI Qian
Subject: Re: Can't exclude unnecessary pages for 2.6.31 Kernel
Date: Fri, 02 Oct 2009 19:26:44 +0800 (CST)
> Cced the crash utility maintainer.
>
> BZ filed -- https://bugzilla.redhat.com/show_bug.cgi?id=526749
>
>> It does make -d 16 works without errors. However, the crash utility
>> failed to read it (the log attached), but the full VMCore is working
>> fine there.
>>
>> crash: page excluded: kernel virtual address: 819f1764 type:
>>"tss_struct ist array"
Not sure if it is related, but kmem -z fails with the full VMCore.
crash> kmem -z
kmem: invalid (optional) structure member offsets: zone_pages_min or
zone_struct_pages_min FILE: memory.c
LINE: 12217 FUNCTION: dump_zone_stats()
[/usr/bin/crash] error trace: 44bb25 => 4701fc => 46efa6 => 4d4c57
4d4c57: OFFSET_option+167
46efa6: (undetermined)
4701fc: cmd_kmem+3212
44bb25: exec_command+613
kmem: invalid (optional) structure member offsets: zone_pages_min or
zone_struct_pages_min
FILE: memory.c LINE: 12217 FUNCTION: dump_zone_stats()
>
> Thanks!
> CAI Qian
>
> From: "Ken'ichi Ohmichi"
> Subject: Re: Can't exclude unnecessary pages for 2.6.31 Kernel
> Date: Fri, 02 Oct 2009 09:16:50 +0900
>
>>
>> Hi CAI,
>>
>> Thank you for testing and your report.
>>
>> CAI Qian wrote:
>>> Met 2 issues with 2.6.31 kernel.
>>>
>>> (1) makedumpfile needs to be updated to support 2.6.31 kernel. This looks
>>> like
>>> an easy fix. In makedumpfile.h,
>>>
>>> #define LATEST_VERSION (0x0206001d)/* linux-2.6.29 */
>>
>> The next makedumpfile will support 2.6.31 kernel, and the above will be
>> fixed.
>>
>>
>>> (2) makedumpfile -d 16 failed.
>>>
>>> # makedumpfile -D -d 16
>>> /var/crash/127.0.0.1-2009-10-01-06\:22\:33/vmcore vmcore
>>>
>>> LOAD (0)
>>> phys_start : 100
>>> phys_end : 2683000
>>> virt_start : 8100
>>> virt_end : 82683000
>>> LOAD (1)
>>> phys_start : 0
>>> phys_end : a
>>> virt_start : 8100
>>> virt_end : 810a
>>> LOAD (2)
>>> phys_start : 10
>>> phys_end : 300
>>> virt_start : 8110
>>> virt_end : 81000300
>>> LOAD (3)
>>> phys_start : b00
>>> phys_end : 3ff7
>>> virt_start : 81000b00
>>> virt_end : 81003ff7
>>> Linux kdump
>>> page_size: 4096
>>>
>>> max_mapnr: 3ff70
>>> The kernel version is not supported.
>>> The created dumpfile may be incomplete.
>>>
>>> num of NODEs : 1
>>>
>>>
>>> Memory type : SPARSEMEM
>>>
>>> mem_map (0)
>>> mem_map: 880002664000
>>> pfn_start : 0
>>> pfn_end: 8000
>>> mem_map (1)
>>> mem_map: 0
>>> pfn_start : 8000
>>> pfn_end: 1
>>> mem_map (2)
>>> mem_map: 0
>>> pfn_start : 1
>>> pfn_end: 18000
>>> mem_map (3)
>>> mem_map: 0
>>> pfn_start : 18000
>>> pfn_end: 2
>>> mem_map (4)
>>> mem_map: 0
>>> pfn_start : 2
>>> pfn_end: 28000
>>> mem_map (5)
>>> mem_map: 0
>>> pfn_start : 28000
>>> pfn_end: 3
>>> mem_map (6)
>>> mem_map: 0
>>> pfn_start : 3
>>> pfn_end: 38000
>>> mem_map (7)
>>> mem_map: 0
>>> pfn_start : 38000
>>> pfn_end: 3ff70
>>> Excluding unnecessary pages: [100 %] page_to_pfn: Can't convert
>>> the address of page descriptor (ea04e000) to pfn.
>>>
>>> create_2nd_bitmap: Can't exclude unnecessary pages.
>>>
>>> makedumpfile Failed.
>>
>> The cause is that the virtual memory map of x86_64 has been changed to
>> the following since linux-2.6.31.
>>
>> 15250 --- a/Documentation/x86/x86_64/mm.txt
>> 15251 +++ b/Documentation/x86/x86_64/mm.txt
>> 15252 @@ -6,10 +6,11 @@ Virtual memory map with 4 level page tables:
>> 15253 - 7fff (=47 bits) user space, different
>> per mm
>> 15254 hole caused by [48:63] sign extension
>> 15255 8000 - 80ff (=40
Re: Can't exclude unnecessary pages for 2.6.31 Kernel
Cced the crash utility maintainer. BZ filed -- https://bugzilla.redhat.com/show_bug.cgi?id=526749 > It does make -d 16 works without errors. However, the crash utility > failed to read it (the log attached), but the full VMCore is working > fine there. > > crash: page excluded: kernel virtual address: 819f1764 type: >"tss_struct ist array" Thanks! CAI Qian From: "Ken'ichi Ohmichi" Subject: Re: Can't exclude unnecessary pages for 2.6.31 Kernel Date: Fri, 02 Oct 2009 09:16:50 +0900 > > Hi CAI, > > Thank you for testing and your report. > > CAI Qian wrote: >> Met 2 issues with 2.6.31 kernel. >> >> (1) makedumpfile needs to be updated to support 2.6.31 kernel. This looks >> like >> an easy fix. In makedumpfile.h, >> >> #define LATEST_VERSION (0x0206001d)/* linux-2.6.29 */ > > The next makedumpfile will support 2.6.31 kernel, and the above will be fixed. > > >> (2) makedumpfile -d 16 failed. >> >> # makedumpfile -D -d 16 >> /var/crash/127.0.0.1-2009-10-01-06\:22\:33/vmcore vmcore >> >> LOAD (0) >> phys_start : 100 >> phys_end : 2683000 >> virt_start : 8100 >> virt_end : 82683000 >> LOAD (1) >> phys_start : 0 >> phys_end : a >> virt_start : 8100 >> virt_end : 810a >> LOAD (2) >> phys_start : 10 >> phys_end : 300 >> virt_start : 8110 >> virt_end : 81000300 >> LOAD (3) >> phys_start : b00 >> phys_end : 3ff7 >> virt_start : 81000b00 >> virt_end : 81003ff7 >> Linux kdump >> page_size: 4096 >> >> max_mapnr: 3ff70 >> The kernel version is not supported. >> The created dumpfile may be incomplete. >> >> num of NODEs : 1 >> >> >> Memory type : SPARSEMEM >> >> mem_map (0) >> mem_map: 880002664000 >> pfn_start : 0 >> pfn_end: 8000 >> mem_map (1) >> mem_map: 0 >> pfn_start : 8000 >> pfn_end: 1 >> mem_map (2) >> mem_map: 0 >> pfn_start : 1 >> pfn_end: 18000 >> mem_map (3) >> mem_map: 0 >> pfn_start : 18000 >> pfn_end: 2 >> mem_map (4) >> mem_map: 0 >> pfn_start : 2 >> pfn_end: 28000 >> mem_map (5) >> mem_map: 0 >> pfn_start : 28000 >> pfn_end: 3 >> mem_map (6) >> mem_map: 0 >> pfn_start : 3 >> pfn_end: 38000 >> mem_map (7) >> mem_map: 0 >> pfn_start : 38000 >> pfn_end: 3ff70 >> Excluding unnecessary pages: [100 %] page_to_pfn: Can't convert >> the address of page descriptor (ea04e000) to pfn. >> >> create_2nd_bitmap: Can't exclude unnecessary pages. >> >> makedumpfile Failed. > > The cause is that the virtual memory map of x86_64 has been changed to > the following since linux-2.6.31. > > 15250 --- a/Documentation/x86/x86_64/mm.txt > 15251 +++ b/Documentation/x86/x86_64/mm.txt > 15252 @@ -6,10 +6,11 @@ Virtual memory map with 4 level page tables: > 15253 - 7fff (=47 bits) user space, different > per mm > 15254 hole caused by [48:63] sign extension > 15255 8000 - 80ff (=40 bits) guard hole > 15256 -8800 - c0ff (=57 TB) direct mapping of all > phys. memory > 15257 -c100 - c1ff (=40 bits) hole > 15258 -c200 - e1ff (=45 bits) vmalloc/ioremap space > 15259 -e200 - e2ff (=40 bits) virtual memory map > (1TB) > 15260 +8800 - c7ff (=64 TB) direct mapping of all > phys. memory > 15261 +c800 - c8ff (=40 bits) hole > 15262 +c900 - e8ff (=45 bits) vmalloc/ioremap space > 15263 +e900 - e9ff (=40 bits) hole > 15264 +ea00 - eaff (=40 bits) virtual memory map > (1TB) > > I created the patch for catching this change. > Could you try the attached patch ? > > > Thanks > Ken'ichi Ohmichi > > --- > [PATCH] Add x86_64 linux-2.6.31 support. > > Virtual memory map of x86_64 has been changed to the following > since linux-2.6.31. > > 15250 --- a/Documentation/x86/x86_64/mm.txt > 15251 +++ b/Documentation/x86/x86_64/mm.txt > 15252 @@ -6,10 +6,11 @@ Virtual memory map with
Re: Can't exclude unnecessary pages for 2.6.31 Kernel
Hallo Ken'ichi! From: "Ken'ichi Ohmichi" Subject: Re: Can't exclude unnecessary pages for 2.6.31 Kernel Date: Fri, 02 Oct 2009 09:16:50 +0900 > > The next makedumpfile will support 2.6.31 kernel, and the above will be fixed. > Great. > > I created the patch for catching this change. > Could you try the attached patch ? > It does make -d 16 works without errors. However, the crash utility failed to read it (the log attached), but the full VMCore is working fine there. crash: page excluded: kernel virtual address: 819f1764 type: "tss_struct ist array" Thanks! CAI Qian [r...@nec-em18 analyse-crash]# crash -d 3 /usr/lib/debug/lib/modules/2.6.31-23.el6.x86_64/vmlinux /var/crash/16/vmcore.fail crash 4.0.9 Copyright (C) 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009 Red Hat, Inc. Copyright (C) 2004, 2005, 2006 IBM Corporation Copyright (C) 1999-2006 Hewlett-Packard Co Copyright (C) 2005, 2006 Fujitsu Limited Copyright (C) 2006, 2007 VA Linux Systems Japan K.K. Copyright (C) 2005 NEC Corporation Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc. Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc. This program is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Enter "help copying" to see the conditions. This program has absolutely no warranty. Enter "help warranty" for details. compressed kdump: header->utsname.machine: diskdump_data: filename: (null) flags: 6 (KDUMP_CMPRS_LOCAL|ERROR_EXCLUDED) dfd: 3 ofp: 0 machine_type: 62 (EM_X86_64) header: bb4c70 signature: "KDUMP " header_version: 2 utsname: sysname: nodename: release: version: machine: domainname: timestamp: tv_sec: 4ac44a6f tv_usec: 0 status: 0 () block_size: 4096 sub_hdr_size: 1 bitmap_blocks: 16 max_mapnr: 262000 total_ram_blocks: 0 device_blocks: 0 written_blocks: 0 current_cpu: 0 nr_cpus: 1 tasks[nr_cpus]: 0 sub_header: 0 (n/a) sub_header_kdump: bb5c80 phys_base: 0 dump_level: 16 (0x10) (DUMP_EXCLUDE_FREE) data_offset: 12000 block_size: 4096 block_shift: 12 bitmap: bb6c90 bitmap_len: 65536 dumpable_bitmap: bc6ca0 byte: 0 bit: 0 compressed_page: be6ed0 curbufptr: 0 page_cache_hdr[0]: pg_flags: 0 () pg_addr: 0 pg_bufptr: bd6ec0 pg_hit_count: 0 page_cache_hdr[1]: pg_flags: 0 () pg_addr: 0 pg_bufptr: bd7ec0 pg_hit_count: 0 page_cache_hdr[2]: pg_flags: 0 () pg_addr: 0 pg_bufptr: bd8ec0 pg_hit_count: 0 page_cache_hdr[3]: pg_flags: 0 () pg_addr: 0 pg_bufptr: bd9ec0 pg_hit_count: 0 page_cache_hdr[4]: pg_flags: 0 () pg_addr: 0 pg_bufptr: bdaec0 pg_hit_count: 0 page_cache_hdr[5]: pg_flags: 0 () pg_addr: 0 pg_bufptr: bdbec0 pg_hit_count: 0 page_cache_hdr[6]: pg_flags: 0 () pg_addr: 0 pg_bufptr: bdcec0 pg_hit_count: 0 page_cache_hdr[7]: pg_flags: 0 () pg_addr: 0 pg_bufptr: bddec0 pg_hit_count: 0 page_cache_hdr[8]: pg_flags: 0 () pg_addr: 0 pg_bufptr: bdeec0 pg_hit_count: 0 page_cache_hdr[9]: pg_flags: 0 () pg_addr: 0 pg_bufptr: bdfec0 pg_hit_count: 0 page_cache_hdr[10]: pg_flags: 0 () pg_addr: 0 pg_bufptr: be0ec0 pg_hit_count: 0 page_cache_hdr[11]: pg_flags: 0 () pg_addr: 0 pg_bufptr: be1ec0 pg_hit_count: 0 page_cache_hdr[12]: pg_flags: 0 () pg_addr: 0 pg_bufptr: be2ec0 pg_hit_count: 0 page_cache_hdr[13]: pg_flags: 0 () pg_addr: 0 pg_bufptr: be3ec0 pg_hit_count: 0 page_cache_hdr[14]: pg_flags: 0 () pg_addr: 0 pg_bufptr: be4ec0 pg_hit_count: 0 page_cache_hdr[15]: pg_flags: 0 () pg_addr: 0 pg_bufptr: be5ec0 pg_hit_count: 0 page_cache_buf: bd6ec0 evict_index: 0 evictions: 0 accesses: 0 cached_reads: 0 valid_pages: bd6cb0 crash: pv_init_ops exists: ARCH_PVOPS compressed kdump: phys_base: 0 gdb /usr/lib/debug/lib/modules/2.6.31-23.
Re: Can't exclude unnecessary pages for 2.6.31 Kernel
From: "Ken'ichi Ohmichi" Subject: Re: Can't exclude unnecessary pages for 2.6.31 Kernel Date: Fri, 02 Oct 2009 16:30:37 +0900 > > Hi CAI, > > CAI Qian wrote: >>>>> I created the patch for catching this change. >>>>> Could you try the attached patch ? >>>>> >>>> It does make -d 16 works without errors. However, the crash utility >>>> failed to read it (the log attached), but the full VMCore is working >>>> fine there. >>>> >>>> crash: page excluded: kernel virtual address: 819f1764 type: >>>>"tss_struct ist array" >>> I can not reproduce the above problem, and I want to reproduce this >>> problem on my machine for investigating it. >>> >>> Could you send your kernel .config file ? >> >> Sure. Attached. > > Thank you. > I'm making a linux-2.6.31 kernel by your .config file. > > >> Not sure if this the crash utility problem or not, FYI >> it was reproduced using the latest upstream version. > > Is "the latest upstream version" 2.6.32-rc1 ? Sorry, I have not made my self clearly. I mean the latest upstream version of the crash utility. http://people.redhat.com/anderson/ The kernel is 2.6.31. Thanks! CAI Qian > > > Thanks > Ken'ichi Ohmichi ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Can't exclude unnecessary pages for 2.6.31 Kernel
Hallo! Met 2 issues with 2.6.31 kernel. (1) makedumpfile needs to be updated to support 2.6.31 kernel. This looks like an easy fix. In makedumpfile.h, #define LATEST_VERSION (0x0206001d)/* linux-2.6.29 */ (2) makedumpfile -d 16 failed. # makedumpfile -D -d 16 /var/crash/127.0.0.1-2009-10-01-06\:22\:33/vmcore vmcore LOAD (0) phys_start : 100 phys_end : 2683000 virt_start : 8100 virt_end : 82683000 LOAD (1) phys_start : 0 phys_end : a virt_start : 8100 virt_end : 810a LOAD (2) phys_start : 10 phys_end : 300 virt_start : 8110 virt_end : 81000300 LOAD (3) phys_start : b00 phys_end : 3ff7 virt_start : 81000b00 virt_end : 81003ff7 Linux kdump page_size: 4096 max_mapnr: 3ff70 The kernel version is not supported. The created dumpfile may be incomplete. num of NODEs : 1 Memory type : SPARSEMEM mem_map (0) mem_map: 880002664000 pfn_start : 0 pfn_end: 8000 mem_map (1) mem_map: 0 pfn_start : 8000 pfn_end: 1 mem_map (2) mem_map: 0 pfn_start : 1 pfn_end: 18000 mem_map (3) mem_map: 0 pfn_start : 18000 pfn_end: 2 mem_map (4) mem_map: 0 pfn_start : 2 pfn_end: 28000 mem_map (5) mem_map: 0 pfn_start : 28000 pfn_end: 3 mem_map (6) mem_map: 0 pfn_start : 3 pfn_end: 38000 mem_map (7) mem_map: 0 pfn_start : 38000 pfn_end: 3ff70 Excluding unnecessary pages: [100 %] page_to_pfn: Can't convert the address of page descriptor (ea04e000) to pfn. create_2nd_bitmap: Can't exclude unnecessary pages. makedumpfile Failed. Thanks! CAI Qian ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: Kdump Automation Mechanism
Hi Jay, From: Jay Lan <[EMAIL PROTECTED]> Subject: Re: Kdump Automation Mechanism Date: Wed, 20 Aug 2008 13:07:53 -0700 > Cai Qian wrote: > > Hi Bernhard and Neil, > > > > --- Bernhard Walle <[EMAIL PROTECTED]> wrote: > > > >> * Jay Lan [2008-08-14 10:23]: > >>> If you can put vmcore at the same location... Ahh, asking too much? ;) > >> Will change that to /var/crash since that's FHS 2.3 anyway. :) > >> [And I personally never liked “/var/log/dump†since it's not a > >> logfile.] > >> > > > > That's a good. Do you have any consideration for standard Kdump options? I > > have created a Kdump > > test suite but it originally written and tested on RHEL and Fedora. > > > > http://sourceforge.net/mailarchive/message.php?msg_name=20080815.210735.450985660335275199.qcai%40redhat.com > > Hi Cai, > > I clicked on the attachment of the page above, it failed with: >File not found on server. > Then, please find it at, http://people.redhat.com/qcai/kdump-ng-1.0.tar.gz Thanks, CaiQian > Regards, > - jay > > > > > I hope it could also work for SLES, OpenSUSE and other distros. However, I > > have no environment to > > test it yet, and I found it is kind of difficult to port it to other > > distros for so many different > > options. > > > > I'd appreciate if you guys could have a look, and come out a standard Kdump > > options. I'd also be > > happy to modify the suite accordingly. > > > > Thanks, > > CaiQian > > > >> > >> Bernhard > >> -- > >> Bernhard Walle, SUSE LINUX Products GmbH, Architecture Development > >> > >> “Make everything as simple as possible, but not simpler.†> >>-- Albert Einstein > >> > >> ___ > >> kexec mailing list > >> kexec@lists.infradead.org > >> http://lists.infradead.org/mailman/listinfo/kexec > >> > > > > > > ___ > > kexec mailing list > > kexec@lists.infradead.org > > http://lists.infradead.org/mailman/listinfo/kexec > > > ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: Kdump Automation Mechanism
Hi Bernhard and Neil, --- Bernhard Walle <[EMAIL PROTECTED]> wrote: > * Jay Lan [2008-08-14 10:23]: > > > > If you can put vmcore at the same location... Ahh, asking too much? ;) > > Will change that to /var/crash since that's FHS 2.3 anyway. :) > [And I personally never liked â/var/log/dumpâ since it's not a logfile.] > That's a good. Do you have any consideration for standard Kdump options? I have created a Kdump test suite but it originally written and tested on RHEL and Fedora. http://sourceforge.net/mailarchive/message.php?msg_name=20080815.210735.450985660335275199.qcai%40redhat.com I hope it could also work for SLES, OpenSUSE and other distros. However, I have no environment to test it yet, and I found it is kind of difficult to port it to other distros for so many different options. I'd appreciate if you guys could have a look, and come out a standard Kdump options. I'd also be happy to modify the suite accordingly. Thanks, CaiQian > > > Bernhard > -- > Bernhard Walle, SUSE LINUX Products GmbH, Architecture Development > > âMake everything as simple as possible, but not simpler.â >-- Albert Einstein > > ___ > kexec mailing list > kexec@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/kexec > ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Kdump Automation Mechanism
Hi, I am wondering if this is the right project to accept Kdump automation files. For example, Kdump daemon init script, mkdumprd (read from Kdump configuration file and generate Kdump initramfs for the system), default Kdump configuration file, tools to compress vmcore etc. The problem I am having right now is that, I am trying to write a Kdump test suite, but I could not find a easy way to write a general enough suite for every distro to use. Red Hat and SUSE have slightly different setting up. Debian and Ubuntu don't have Kdump automation mechanism as the time I was checking. Also, It make things much easier for different distros and users to pick it up and contribute to those efforts. Thanks, CaiQian ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [LTP] [RFC] [KDUMP] [PROPOSED WORK] kdump on Xen hypervisor and guests, more tests for utilities, like makedumpfile, mkdumprd, kexec etc
Hi Subrata, From: Subrata Modak <[EMAIL PROTECTED]> Subject: Re: [LTP] [RFC] [KDUMP] [PROPOSED WORK] kdump on Xen hypervisor and guests, more tests for utilities, like makedumpfile, mkdumprd, kexec etc Date: Tue, 12 Aug 2008 11:43:02 +0530 > On Tue, 2008-08-12 at 10:16 +0800, Cai Qian wrote: > > Hi Subrata, > > > > From: Subrata Modak <[EMAIL PROTECTED]> > > Subject: Re: [LTP] [RFC] [KDUMP] [PROPOSED WORK] kdump on Xen hypervisor > > and guests, more tests for utilities, like makedumpfile, mkdumprd, kexec etc > > Date: Mon, 11 Aug 2008 19:02:47 +0530 > > > > > Cai, > > > > > > Any headway in this front ? > > > > > > > I have all test cases finished, but I am working on implementing a > > suitable harness to run those tests. > > Thanks Cai. Happy to know that you are done with all the test cases, and > only the integration part is pending. I am little bit confused when you > say that you are working on implementing a suitable harness to run those > tests. Does that mean that the way the present LTP-Kdump tests are run > is not suitable to accommodate the enhancements/additions that you are > doing to the LTP-Kdump ? > What I have already done is to create several basic building blocks for testing of Kdump. For example, setup-bare-metal, config-ext3, crash-sysrq-c, crash-lkdtm, analyse-crash etc. You probably could guess what they are doing from their names. Then, we could combine those building blocks to create test cases, which should give us more freedom and flexibility to test things. I am going to include some pre-defined test cases into LTP as well. I might propose new tests as LTP-kdump-ng. > I do not mind in having a nice harness within LTP for executing the > LTP-Kdump tests, but i would hope that the new harness should also > include the existing LTP-Kdump tests, so that you use the same (new > harness) to tests everything in LTP-Kdump. Let me know what you think > for the same. > The problem that I am trying to figure it out is that we need a suitable harness to schedule those tests and report results. For example, there is a test case is like the following, setup-bare-metal config-ext3:uuid crash-sysrq-c analyse-crash The first and the third steps requires to reboot the machine, so we need a harness could resume execution of tests after reboot. I am thinking of using cron job as in existing LTP-kdump tests, or a tests injector running as a daemon, or something else. New tests should incorporate all existing tests and coverage of Kdump in LTP. CaiQian > Regards-- > Subrata > > > > > CaiQian > > > > > Regards-- > > > Subrata > > > > > > On Thu, 2008-07-24 at 05:11 +0530, Subrata Modak wrote: > > > > Hi Cai, > > > > > > > > Some patches in Bits and Pieces regarding this nearby ? > > > > > > > > Regards-- > > > > Subrata > > > > > > > > > > > > > > Yes. Initally, I put this item to a relative low priority, partly > > > > > because kdump config options and init scripts are tend to be > > > > > distro-specific, so it won't be easy to write portable tests for > > > > > different distros. In addition, lots of config options are not easy to > > > > > be tested automately, like raw disk target, vfat disk target, and > > > > > network target etc, as testers have to setup those name manually. But, > > > > > you are right, those are high priority tests for kexec/kdump in distro > > > > > release, we tested most of those options manually for RHEL anyway and > > > > > we > > > > > had some automated tests of it, which I'll try to submit to LTP as > > > > > many > > > > > as possible. For those manual tests, I'll also try to find some ways > > > > > to > > > > > automate them. For example, for different dump targets, it is possible > > > > > to verify the generated initrd file as expected. > > > > > > > > > > > > > > > > > > == increase coverages for new kexec/kdump development efforts == > > > > > > > - new reserved region syntax in Kernel. > > > > > > > > > > > > Another important thing we need to focus on is driver testing. > > > > > > Drivers > > > > > > can fail to initialize in second kernel and kdump will fail. Can we > > > > > > do > > > > > > something so that we can do following. > > > > > > > > >
Re: [LTP] [RFC] [KDUMP] [PROPOSED WORK] kdump on Xen hypervisor and guests, more tests for utilities, like makedumpfile, mkdumprd, kexec etc
Hi Subrata, From: Subrata Modak <[EMAIL PROTECTED]> Subject: Re: [LTP] [RFC] [KDUMP] [PROPOSED WORK] kdump on Xen hypervisor and guests, more tests for utilities, like makedumpfile, mkdumprd, kexec etc Date: Mon, 11 Aug 2008 19:02:47 +0530 > Cai, > > Any headway in this front ? > I have all test cases finished, but I am working on implementing a suitable harness to run those tests. CaiQian > Regards-- > Subrata > > On Thu, 2008-07-24 at 05:11 +0530, Subrata Modak wrote: > > Hi Cai, > > > > Some patches in Bits and Pieces regarding this nearby ? > > > > Regards-- > > Subrata > > > > > > > > Yes. Initally, I put this item to a relative low priority, partly > > > because kdump config options and init scripts are tend to be > > > distro-specific, so it won't be easy to write portable tests for > > > different distros. In addition, lots of config options are not easy to > > > be tested automately, like raw disk target, vfat disk target, and > > > network target etc, as testers have to setup those name manually. But, > > > you are right, those are high priority tests for kexec/kdump in distro > > > release, we tested most of those options manually for RHEL anyway and we > > > had some automated tests of it, which I'll try to submit to LTP as many > > > as possible. For those manual tests, I'll also try to find some ways to > > > automate them. For example, for different dump targets, it is possible > > > to verify the generated initrd file as expected. > > > > > > > > > > > > == increase coverages for new kexec/kdump development efforts == > > > > > - new reserved region syntax in Kernel. > > > > > > > > Another important thing we need to focus on is driver testing. Drivers > > > > can fail to initialize in second kernel and kdump will fail. Can we do > > > > something so that we can do following. > > > > > > > > > > That isn't something I have not thought of. For RHEL release testing, we > > > will have a workflow to run those tests on any storage/network drivers, > > > and it will report back testing results and driver matrix. However, this > > > workflow is very distro-specific, and depends on the test farm it is > > > using, so it does not make any sense to put it here. > > > > > > > - Collect the machine statistics on which kdump was tested and send > > > > the reports to a common place. Especially capture the storage/network > > > > driver data which can be probably be available through LTP site. > > > > > > > > > > That sounds like a good idea to me. > > > > > > > - Also capture how much memory was reserved on what architecture and > > > > whether it worked or not. This will help us verify for sure that how > > > > much memory to reserve for second kernel on various architectures. > > > > > > > > > > This is something could be done. I'll have a look at it. > > > > > > Thanks, > > > CaiQian > > > > > > > Thanks > > > > Vivek > > > > > > - > > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > > Build the coolest Linux based applications with Moblin SDK & win great > > prizes > > Grand prize is a trip for two to an Open Source event anywhere in the world > > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > > ___ > > Ltp-list mailing list > > [EMAIL PROTECTED] > > https://lists.sourceforge.net/lists/listinfo/ltp-list > ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [RFC] [KDUMP] [PROPOSED WORK] kdump on Xen hypervisor and guests, more tests for utilities, like makedumpfile, mkdumprd, kexec etc
Hi, From: Vivek Goyal <[EMAIL PROTECTED]> Subject: Re: [RFC] [KDUMP] [PROPOSED WORK] kdump on Xen hypervisor and guests, more tests for utilities, like makedumpfile, mkdumprd, kexec etc Date: Tue, 24 Jun 2008 08:42:43 -0400 > On Mon, Jun 23, 2008 at 07:42:50PM +0530, Subrata Modak wrote: > > Hi, > > > > Cai has proposed to work on the above LTP-KDUMP test cases > > enrichment/enhancements. Please let us know about your views on the > > same. We encourage people to review his proposal and the corresponding > > upcoming test cases. I am going to put this soon on the LTP-KDUMP plan > > document. > > > > http://ltp.cvs.sourceforge.net/ltp/ltp/testcases/kdump/doc/TEST_PLAN.txt, > > > > Hi Subrata/Cai, > > That's a very good idea. We need to increase kdump test coverage and > automate the whole thing. > > > .. > > > > Here is my first draft plan of Kexec/Kdump tests enhancement sorted by > > priorities. I would like to add them as many as possible. > > > > == filtered vmcore utilities == > > - in different compressed levels, verify the vmcore with the correct > > layout. > > - verify it in flat file or ELF formats from a network host. > > > > == analyse vmcore utilities == > > - GDB > > - crash with better error detecting. > > - crash to analyse Hypervisor and Dom0 Kernel. > > > > == test scripts == > > - timestamp information for crash was triggered, vmcore was generated, > > and vmcore was verified. > > - aim to 100% automation, and reduce manual setup. > > - tidy up scripts. > > > > == crash scenarios == > > - SDINT switch for ia64 if possible. > > - Hypervisor crash for Virtualization. > > - crashes on full- and para-virt guests. > > > > == fix bugs in existing tests == > > - printk LKDTM module can hang the second Kernel. > > > > == kdump configurations and init script == > > - capture vmcore after init runs. > > - rpm pre- and post-scripts > > - kdump_pre and kdump_post directives > > > > Can we boost the priority of this item. Making sure all the > kdump config options are working as stated. This is the interface > a kdump user first sees and if it does not work, then it leaves a very > bad impression. Yes. Initally, I put this item to a relative low priority, partly because kdump config options and init scripts are tend to be distro-specific, so it won't be easy to write portable tests for different distros. In addition, lots of config options are not easy to be tested automately, like raw disk target, vfat disk target, and network target etc, as testers have to setup those name manually. But, you are right, those are high priority tests for kexec/kdump in distro release, we tested most of those options manually for RHEL anyway and we had some automated tests of it, which I'll try to submit to LTP as many as possible. For those manual tests, I'll also try to find some ways to automate them. For example, for different dump targets, it is possible to verify the generated initrd file as expected. > > > == increase coverages for new kexec/kdump development efforts == > > - new reserved region syntax in Kernel. > > Another important thing we need to focus on is driver testing. Drivers > can fail to initialize in second kernel and kdump will fail. Can we do > something so that we can do following. > That isn't something I have not thought of. For RHEL release testing, we will have a workflow to run those tests on any storage/network drivers, and it will report back testing results and driver matrix. However, this workflow is very distro-specific, and depends on the test farm it is using, so it does not make any sense to put it here. > - Collect the machine statistics on which kdump was tested and send > the reports to a common place. Especially capture the storage/network > driver data which can be probably be available through LTP site. > That sounds like a good idea to me. > - Also capture how much memory was reserved on what architecture and > whether it worked or not. This will help us verify for sure that how > much memory to reserve for second kernel on various architectures. > This is something could be done. I'll have a look at it. Thanks, CaiQian > Thanks > Vivek ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
