From: Cliff Wickman
Applies to the development branch as of 10/13/2015.
Incorporates review 10/22 by kumagai-atsushi.
Incorporates review 10/26 by kumagai-atsushi. (arch x86_64-only implementation)
This patch adds a -e option to makedumpfile.
The -e option excludes kernel pages that contain
From: Cliff Wickman
Applies to the development branch as of 10/13/2015.
Incorporates review 10/22 by kumagai-atsushi.
This patch adds a -e option to makedumpfile.
The -e option excludes kernel pages that contain nothing but kernel page
structures for pages that are not being included in the
From: Cliff Wickman
Applies to the development branch as of 10/13/2015.
This patch adds a -e option to makedumpfile.
The -e option excludes kernel pages that contain nothing but kernel page
structures for pages that are not being included in the dump.
The -e option only works in non-cyclic mode
From: Cliff Wickman
This patch has been submitted before (see Jun29 2015), but as part of a
2-patch set. That set included a direct i/o option, but that idea has been
dropped as unnecessary.
Also submitted on Aug28, and those review comments incorporated.
This patch applies after patch
From: Cliff Wickman
This patch enhances the --work-dir feature to make it easier to use.
Currently the --work-dir= must be specified in a form that is valid in
the crashkernel's environment.
In other words, the boot kernel's /var, for example, must be specified as:
RHEL6: /mnt/va
On Thu, Sep 10, 2015 at 07:44:02AM +, Atsushi Kumagai wrote:
> >Hello Kumagai,
> >
> >On Wed, Sep 09, 2015 at 07:51:50AM +, Atsushi Kumagai wrote:
> >> Hello Cliff,
> >>
> >> >From: Cliff Wickman
> >> >
> >>
Hello Kumagai,
On Wed, Sep 09, 2015 at 07:51:50AM +, Atsushi Kumagai wrote:
> Hello Cliff,
>
> >From: Cliff Wickman
> >
> >This patch enhances the --work-dir feature to make it easier to use.
> >
> >Kumagai, I plan to require the --work-dir for the -e f
le processing, but I don't understand
> >> the actual benefit yet because page cache is reclaimable and it's
> >> generally usable. Does it practically affect the minimum size of
> >> crashkernel= which makedumpfile can work on ?
> >>
> >&
unds reasonable. However, even from the view point of
> performance, page cached I/O is better than direct I/O according to your
> test results.
>
> Please explain the practical benefit of Direct I/O, otherwise I can't
> decide to accept this.
>
>
> T
From: Cliff Wickman
Applies to version 1.5.8
This patch adds a -j to makedumpfile. With this option it uses direct i/o on
the dump
file and the bitmap file, thus enabling makedumpfile to run mode in a fairly
small
crashkernel area without using cyclic mode. It can dump a system with many
From: Cliff Wickman
Applies to version 1.5.8
This patch adds a -e option to makedumpfile.
The -e option excludes kernel pages that contain nothing but kernel page
structures
for pages that are not being included in the dump.
A page structure (56 bytes) exists for every 4096-byte page.
This
Hi Mr. Kumagai,
On Mon, Dec 15, 2014 at 02:33:58AM +, Atsushi Kumagai wrote:
> >On Thu, Dec 11, 2014 at 06:34:32AM +, Atsushi Kumagai wrote:
> >> Hello Cliff,
> >>
> >> >From: Cliff Wickman
> >> >
> >> >This patch adds a -j to
On Thu, Dec 11, 2014 at 06:34:32AM +, Atsushi Kumagai wrote:
> Hello Cliff,
>
> >From: Cliff Wickman
> >
> >This patch adds a -j to makedumpfile. With this option it uses direct i/o on
> >the dump
> >file and the bitmap file, thus enabling makedumpfi
From: Cliff Wickman
This patch adds a -e option to makedumpfile.
The -e option excludes kernel pages that contain nothing but kernel page
structures
for pages that are not being included in the dump.
A page structure (56 bytes) exists for every 4096-byte page.
This amounts to 3.67 million
From: Cliff Wickman
This patch adds a -j to makedumpfile. With this option it uses direct i/o on
the dump
file and the bitmap file, thus enabling makedumpfile to run mode in a fairly
small
crashkernel area without using cyclic mode. It can dump system with many
terabytes of
memory using
From: Cliff Wickman
Back in early 2014 I submitted several patches for your consideration, most of
which concerned saving time and disk space when dumping very large memories.
I offer two of them again, as being the most important elements for the
practical dumping of very large memories
On Mon, Jan 13, 2014 at 10:58:31AM +0100, Michael Holzheu wrote:
> On Fri, 10 Jan 2014 11:58:30 -0600
> Cliff Wickman wrote:
>
> [snip]
>
> > Use O_DIRECT (raw) i/o for the dump and for the bitmaps file, so that
> > writing
> > to those files does not allo
On Fri, Jan 10, 2014 at 07:48:27AM +, Atsushi Kumagai wrote:
> On 2014/01/09 9:26:20, kexec wrote:
> > On Mon, Jan 06, 2014 at 09:27:34AM +, Atsushi Kumagai wrote:
> > > Hello Cliff,
> > >
> > > On 2014/01/01 8:30:47, kexec wrote:
> > > >
From: Cliff Wickman
Version 2:
provides some requested changes:
- remove the automatic exclusion of page structures for memories over 1TB; it
will only be done by explicit request (-e)
- remove the -N option; no need to explicitly include unused vmemmap pages
as they will be included by
es of 4096 bytes.
The kludge is to handle the boundary between the part of the file containing
the page descriptors and the last part of the file, containing the page
data. The data for that boundary area must be assembled into a page buffer and
written with a single write.
Signed-off-by: Cliff Wi
On Mon, Jan 06, 2014 at 09:27:34AM +, Atsushi Kumagai wrote:
> Hello Cliff,
>
> On 2014/01/01 8:30:47, kexec wrote:
> > From: Cliff Wickman
> >
> > Gentlemen of kexec,
> >
> > I have been working on enabling kdump on some very large systems, and
&g
On Thu, Jan 02, 2014 at 11:50:14AM -0500, Dave Anderson wrote:
>
>
> - Original Message -
> > Date: Tue, 31 Dec 2013 17:36:02 -0600
> > From: Cliff Wickman
> > To: kexec@lists.infradead.org, d.hatay...@jp.fujitsu.com,
> > kumagai-atsu...@mxc.nes.
options to exclude or include unneeded
vmemmap pages regardless of system size (see flag_includevm and
flag_excludvm). By default the exclusion of such pages is only
done on a system of a terabyte or more.
Signed-off-by: Cliff Wickma
igned-off-by: Cliff Wickman
---
makedumpfile.c | 501 ++---
makedumpfile.h | 10 +
print_info.c |8
3 files changed, 421 insertions(+), 98 deletions(-)
Index: makedumpfile-1.5.5/makedumpf
On Fri, Aug 30, 2013 at 10:11:09AM +0900, HATAYAMA Daisuke wrote:
> (2013/08/29 7:08), Cliff Wickman wrote:
> > From: Cliff Wickman
> >
> >
> > makedumpfile needs the debug info from either /proc/vmcore or in vmlinux
> > to know how to find free pages using the
On Thu, Aug 29, 2013 at 10:58:37AM +0800, WANG Chao wrote:
> Hi, Cliff
>
> On 08/28/13 at 05:08pm, Cliff Wickman wrote:
> > From: Cliff Wickman
> >
> > Reverse the meanings of -c (compression) and -p (snappy compression) if
> > USESNAPPY is defined.
> >
From: Cliff Wickman
makedumpfile needs the debug info from either /proc/vmcore or in vmlinux
to know how to find free pages using the buddy method.
The distro procedures do not pass the vmlinux (with the -x option), so
add a search for a debug vmlinux in the usual locations.
It's not need
From: Cliff Wickman
If there is plenty of memory use non-cyclic mode.
Diffed against makedumpfile-1.5.4
Signed-off-by: Cliff Wickman
---
makedumpfile.c | 26 ++
Index: makedumpfile-1.5.4/makedumpfile.c
From: Cliff Wickman
Reverse the meanings of -c (compression) and -p (snappy compression) if
USESNAPPY is defined.
The distro kdump scripts seem to only support -c for compression.
So make -c mean snappy compression if it is supported.
Diffed against makedumpfile-1.5.4
Signed-off-by: Cliff
From: Cliff Wickman
- get_mm_sparsemem(): reduce the number of entries in the mem_map[] by
recording only those sections which actually exist in memory
- shorten the executions of __exclude_unnecessary_pages() by passing it only
the pfn's of the current cyclic area
- for testing: add o
From: Cliff Wickman
I am submitting 6 patches that I have found helpful in speeding the dump
process or clarifying the progress report.
They are not a series, and should not be interdependent. But if you
find any dependencies I apply them in this order:
[PATCH] makedumpfile: reverse -c and -p
From: Cliff Wickman
When the kernel free pages are not detectable with the 'buddy' method
we do a scan for free pages.
In cyclic mode that scan was performed over all of memory for each
cycle region.
This patch records the pfn ranges of each node on the first such cycle.
Then ign
From: Cliff Wickman
If there is not enough memory available to run in cyclic mode display
how much more would be needed to do so.
Diffed against makedumpfile-1.5.4
Signed-off-by: Cliff Wickman
---
makedumpfile.c |8 ++--
1 file changed, 6 insertions(+), 2 deletions(-)
Index
I have been testing recent kernel and kexec-tools for doing kdump of large
memories, and found good results.
UV2000 memory: 32TB crashkernel=2G@4G
command line /usr/bin/makedumpfile --non-cylic -c --message-level 23 -d 31 \
--map-size 4096 -x /boot/vmlinux-3
On Mon, May 20, 2013 at 11:20:44AM +0900, Atsushi Kumagai wrote:
> On Wed, 15 May 2013 13:43:59 -0500
> Cliff Wickman wrote:
>
> > From: Cliff Wickman
> >
> > If the vmlinux does not have dwarf information makedumpfile fails in
> > a rather obscure way,
From: Cliff Wickman
Make non-cyclic the default, as it is faster.
Dumping a 1TB memory is successful with a crashkernel=512M.
And allow -Y on the command line, as well as --non-cylic.
Signed-off-by: Cliff Wickman
---
makedumpfile.c |9 ++---
1 file changed, 6 insertions(+), 3
From: Cliff Wickman
If the crash kernel does not support mmap(2) of /proc/vmcore, make
makedumpfile fall back to using reads with a non-alarming message.
Signed-off-by: Cliff Wickman
---
makedumpfile.c | 37 ++---
1 file changed, 22 insertions(+), 15
From: Cliff Wickman
Because addresses of each successive page structure are often descending
it is much faster to use read()'s, so set off info->flag_usemmap
while doing exclude_free_page().
Signed-off-by: Cliff Wickman
---
makedumpfile.c | 15 ++-
1 file changed, 14 in
From: Cliff Wickman
If the vmlinux does not have dwarf information makedumpfile fails in
a rather obscure way, with a flood of redundant errors,
Make it fail with more of a hint of what is wrong.
Signed-off-by: Cliff Wickman
---
dwarf_info.c |4
1 file changed, 4 insertions
From: Cliff Wickman
The 'buddy' identification of free pages should be done for non-cyclical
dumps as well as cyclical.
Remove the cyclic test from this condition:
if (info->flag_cyclic && (info->dump_level & DL_EXCLUDE_FREE))
I find this to speed the scan of page
hem.
(versus Jingbai Ma's results where mmap almost doubled the speed of reads)
I have put counters in to verify, and we are doing several million
seek/read's vs. a few thousand mmap's. Yet the performance is similar
(54sec vs. 37sec, above). I can't rationalize that much improv
roc/vmcore about 80x faster than the
read interface.
That is, doing mmap's and copying data (in pieces the size of page
structures) transfers all of /proc/vmcore about 80 times faster than
reading it.
This greatly speeds up the capture of a kdump, as the scan of page
structures t
On Tue, Apr 23, 2013 at 09:38:57AM +0900, HATAYAMA Daisuke wrote:
> (2013/04/23 2:55), Cliff Wickman wrote:
>> Hello Mr. Atayama and Mr. Kumagai,
>>
>> I have been playing with the v4 patches
>> kdump, vmcore: support mmap() on /proc/vmcore
>> and find t
On Mon, Mar 25, 2013 at 03:42:09PM +0800, Zhang Yanfei wrote:
> ??? 2013???03???05??? 09:51, Simon Horman ??:
> > On Mon, Feb 04, 2013 at 01:53:55PM -0600, Cliff Wickman wrote:
> >> From: Cliff Wickman
> >>
> >> The crash kernel's boot comma
From: Cliff Wickman
On kexec set-up of a crash kernel on a very large memory machine we
sometimes see the worrisome warning:
Too many memory ranges, truncating...
meaning that the total count of e820 ram, reserved and ACPI spaces is over 128.
Per the comment in do_bzImage_load
From: Cliff Wickman
The crash kernel is not able to find its root device if that device is not
on PCI 0.
This is because it is booted with the command line option memmap=exactmap
which currently clears the e820 table and does not restore reserved spaces.
This works for a device on PCI 0
From: Cliff Wickman
The crash kernel's boot command line is not long enough to contain the
necessary memmap= options for a large memory.
The fix is simple, as long as the boot loader's command line is also long
enough.
I'm not sure about boot loader or kernel restrictions
From: Cliff Wickman
The crash kernel is not able to find its root device if that device is not
on PCI 0.
This is because it is booted with the command line option memmap=exactmap
which currently clears the e820 table. So ACPI processing does not
find reserved i/o spaces.
This works for a
An update on testing thise patches:
> This version of the patch improves the consolidation of the mem_map table
> that is passed to the kernel. See make_kernel_mmap().
> Particularly the seemingly duplicate pfn ranges generated on an older
> (2.6.32-based, rhel6) kernel.
On a 2TB machine (idle)
From: Cliff Wickman
This version of the patch improves the consolidation of the mem_map table
that is passed to the kernel. See make_kernel_mmap().
Particularly the seemingly duplicate pfn ranges generated on an older
(2.6.32-based, rhel6) kernel.
I've been experimenting with askin
On Thu, Dec 20, 2012 at 12:22:14PM +0900, HATAYAMA Daisuke wrote:
> From: Cliff Wickman
> Subject: Re: [PATCH] makedumpfile: request the kernel do page scans
> Date: Mon, 10 Dec 2012 09:36:14 -0600
> > On Mon, Dec 10, 2012 at 09:59:29AM +0900, HATAYAMA Daisuke wrote:
> >
From: Cliff Wickman
This patch provides the kernel support for makedumpfile to request
a list of PFNs.
Accompanies
[PATCH v2] makedumpfile: request the kernel do page scans
---
fs/proc/vmcore.c | 570 +++
include/linux/makedumpfile.h
From: Cliff Wickman
I've been experimenting with asking the kernel to scan the page tables
instead of reading all those page structures through /proc/vmcore.
The results are rather dramatic.
On a small, idle UV: about 4 sec. versus about 40 sec.
On a 8TB UV the unnecessary page scan ta
The || in exclude_unnecessary_pages_cyclic() makes makedumpfile
scan all the memmap sections, not just the ones containing the current
cyclic range.
Diffed against makedumpfile-1.5.1
---
makedumpfile.c |2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
Index: makedumpfile-1.5.1/makedumpfi
On Fri, Nov 16, 2012 at 03:39:44PM -0500, Vivek Goyal wrote:
> On Thu, Nov 15, 2012 at 04:52:40PM -0600, Cliff Wickman wrote:
> >
> > Gentlemen,
> >
> > I know this is rather late to the game, given all the recent work to speed
> > up makedumpfile and reduce the
REQUEST_EXCLUDE, asking the kernel
to return lists of PFNs
- adds page scan timing options -n -o and -t
It depends on a kernel patch of course, so I'm also sending one that applies
to a linux-next kernel. See [PATCH] scan page tables for makedumpfile
Diffed against makedumpfile-1.5.0
ested on SGI UV machines, but not yet
proposed to the linux kernel community.
Diffed against linux-next (3.7.0-rc5-next-20121115-uv+)
Signed-off-by: Cliff Wickman
---
fs/proc/vmcore.c | 558 +++
include/linux/makedumpfile.h | 115
2
On Tue, Sep 25, 2012 at 08:10:04AM -0700, Eric W. Biederman wrote:
> Cliff Wickman writes:
>
> > Hi Eric, and all,
> >
> > On Mon, Sep 24, 2012 at 08:11:12PM -0700, Eric W. Biederman wrote:
> >> Cliff Wickman writes:
> >>
> >> > Gentleme
Hi Eric, and all,
On Mon, Sep 24, 2012 at 08:11:12PM -0700, Eric W. Biederman wrote:
> Cliff Wickman writes:
>
> > Gentlemen,
> >
> > In dumping very large memories we are running up against the 896MB
> > limit in SLES11SP2 (3.0.38 kernel).
>
> Odd. That
Gentlemen,
In dumping very large memories we are running up against the 896MB
limit in SLES11SP2 (3.0.38 kernel).
arch/x86/kernel/setup.c
/*
* Keep the crash kernel below this limit. On 32 bits earlier kernels
* would limit the kernel to the low 512 MiB due to mapping restrictions.
* On 64
overflowed. But I suggest doubling the buffer.
Signed-off-by: Cliff Wickman
---
kexec/crashdump.h |4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
Index: kexec-tools-2.0.1/kexec/crashdump.h
===
--- kexec-tools-2.0.
x the BIOS or the kernel, but given
that they may at times generate overlaps, a check in the kexec command
will prevent crash dumps from being effectively disabled.
Diffed against git.kernel.org/pub/scm/linux/kernel/git/horms/kexec-tools.git
Signed-off-by: Cliff Wickman
---
kexec/arch/i386/ke
From: Cliff Wickman
During the reading of /proc/vmcore the kernel is doing ioremap()/iounmap()
repeatedly. And the buildup of un-flushed vm_area_struct's is causing
a great deal of overhead. (rb_next() is chewing up most of that time).
This solution is to provide function set_iounmap_no
From: Cliff Wickman
The copy of /proc/vmcore to a user buffer proceeds much faster
if the kernel addresses memory as cached.
With this patch we have seen an increase in transfer rate from less than
15MB/s to 80-460MB/s, depending on size of the transfer. This makes
a big difference in time
memory is of concern. So there is probably no reason to
change the program logic in this area.
Diffed against kexec-tools-2.0.2
Signed-off-by: Cliff Wickman
---
kexec/crashdump.h |4 ++--
kexec/kexec-elf.c |4
2 files changed, 6 insertions(+), 2 deletions(-)
Index: kexec-tools-2
s generate overlaps, a check in the kexec command
will prevent crash dumps from being effectively disabled.
Diffed against git.kernel.org/pub/scm/linux/kernel/git/horms/kexec-tools.git
Signed-off-by: Cliff Wickman
---
kexec/arch/i386/kexec-x86-common.c | 78 +++
66a6fff : System RAM
766a7000-766d6fff : Unusable memory
...
This patch keeps Unusable memory as another RANGE_RESERVED area, but
without the warning message.
Diffed against git.kernel.org/pub/scm/linux/kernel/git/horms/kexec-tools.git
Signed-off-by: Cliff Wickman
---
kexec/firmware_memmap
LATEST_VERSION is updated to KERNEL_VERSION(2, 6, 32), as this seems to
work on 2.6.32 and prevents a makedumpfile grumbling.
Tested on SGI UV, which is x86_64.
Diffed against makedumpfile-1.3.5
Signed-off-by: Cliff Wickman
---
makedumpfile.h |2 +-
1 file changed, 1 insertion(+), 1
LENAME_BITMAP path should be based at TMPDIR.
Diffed against makedumpfile-1.3.5
Signed-off-by: Cliff Wickman
---
makedumpfile.c | 13 ++---
makedumpfile.h |4 ++--
2 files changed, 12 insertions(+), 5 deletions(-)
Index: makedump
Diffed against git.kernel.org/pub/scm/linux/kernel/git/horms/kexec-tools.git
Signed-off-by: Cliff Wickman
---
kexec/arch/i386/kexec-x86.h |2 +-
kexec/arch/x86_64/crashdump-x86_64.c | 15 +++
2 files changed, 12 insertions(+), 5 deletions(-)
Index: kexec-tools/kexec/arc
the elf header for each cpu.
And the below fiddling with temp_region is just to prevent compiler warnings.
Diffed against kexec-tools-2.0.1
Signed-off-by: Cliff Wickman
---
kexec/arch/i386/kexec-x86.h |2 +-
kexec/arch/x86_64/crashdump-x86_64.c | 19 +++
2 files
ed at TMPDIR.
Also below, LATEST_VERSION is updated to KERNEL_VERSION(2, 6, 32), as this
seems to work on 2.6.32 and prevents a makedumpfile grumbling.
Diffed against makedumpfile-1.3.5
Signed-off-by: Cliff Wickman
---
makedumpfile.c | 13 ++---
makedumpfile.h |4 ++--
2 files changed, 12 i
/arch/x86_64/crashdump-x86_64.c)
Is mine just an extraordinarly large kernel?
-Cliff
--
Cliff Wickman
Silicon Graphics, Inc.
c...@sgi.com
(651) 683-3824
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
On Wed, Aug 27, 2008 at 03:01:57PM -0400, Vivek Goyal wrote:
> On Wed, Aug 27, 2008 at 01:34:13PM -0500, Cliff Wickman wrote:
> >
> > On Wed, Aug 27, 2008 at 11:36:22AM -0400, Vivek Goyal wrote:
> > > On Wed, Aug 27, 2008 at 05:30:35PM +0200, Bernhard Walle wrote:
> &
: Kernel bss
0100-04ff : Crash kernel
cff6-cff68fff : ACPI Tables
cff69000-cff7 : ACPI Non-volatile Storage
cff8-cfff : reserved
d000-d7ff : PCI Bus :08
d000-d7ff : :08:01.0
I do have Bernhard's patch applied:
--- a/arch/x86/kernel/setup_64
00)
cpw: ata_port_wait_eh, finished wait
cpw: ata_port_wait_eh exiting
cpw: ata_host_register done with probe port 1
There is currently some device (udev?) problem with the 2.6.23 kernel. Could
this be related?
-Cliff Wickman
SGI
___
kexec mailing list
76 matches
Mail list logo
