Re: [PATCH v14 01/11] x86: kdump: replace the hard-coded alignment with macro CRASH_ALIGN
On 2021/3/2 15:43, Baoquan He wrote:
> On 02/26/21 at 09:38am, Eric W. Biederman wrote:
>> chenzhou writes:
>>
>>> On 2021/2/25 15:25, Baoquan He wrote:
>>>> On 02/24/21 at 02:19pm, Catalin Marinas wrote:
>>>>> On Sat, Jan 30, 2021 at 03:10:15PM +0800, Chen Zhou wrote:
>>>>>> Move CRASH_ALIGN to header asm/kexec.h for later use. Besides, the
>>>>>> alignment of crash kernel regions in x86 is 16M(CRASH_ALIGN), but
>>>>>> function reserve_crashkernel() also used 1M alignment. So just
>>>>>> replace hard-coded alignment 1M with macro CRASH_ALIGN.
>>>>> [...]
>>>>>> @@ -510,7 +507,7 @@ static void __init reserve_crashkernel(void)
>>>>>> } else {
>>>>>> unsigned long long start;
>>>>>>
>>>>>> -start = memblock_phys_alloc_range(crash_size, SZ_1M,
>>>>>> crash_base,
>>>>>> +start = memblock_phys_alloc_range(crash_size,
>>>>>> CRASH_ALIGN, crash_base,
>>>>>>crash_base +
>>>>>> crash_size);
>>>>>> if (start != crash_base) {
>>>>>> pr_info("crashkernel reservation failed -
>>>>>> memory is in use.\n");
>>>>> There is a small functional change here for x86. Prior to this patch,
>>>>> crash_base passed by the user on the command line is allowed to be 1MB
>>>>> aligned. With this patch, such reservation will fail.
>>>>>
>>>>> Is the current behaviour a bug in the current x86 code or it does allow
>>>>> 1MB-aligned reservations?
>>>> Hmm, you are right. Here we should keep 1MB alignment as is because
>>>> users specify the address and size, their intention should be respected.
>>>> The 1MB alignment for fixed memory region reservation was introduced in
>>>> below commit, but it doesn't tell what is Eric's request at that time, I
>>>> guess it meant respecting users' specifying.
>>
>>> I think we could make the alignment unified. Why is the alignment system
>>> reserved and
>>> user specified different? Besides, there is no document about the 1MB
>>> alignment.
>>> How about adding the alignment size(16MB) in doc if user specified
>>> start address as arm64 does.
>> Looking at what the code is doing. Attempting to reserve a crash region
>> at the location the user specified. Adding unnecessary alignment
>> constraints is totally broken.
>>
>> I am not even certain enforcing a 1MB alignment makes sense. I suspect
>> it was added so that we don't accidentally reserve low memory on x86.
>> Frankly I am not even certain that makes sense.
>>
>> Now in practice there might be an argument for 2MB alignment that goes
>> with huge page sizes on x86. But until someone finds that there are
>> actual problems with 1MB alignment I would not touch it.
>>
>> The proper response to something that isn't documented and confusing is
>> not to arbitrarily change it and risk breaking users. Especially in
>> this case where it is clear that adding additional alignment is total
>> nonsense. The proper response to something that isn't clear and
>> documented is to dig in and document it, or to leave it alone and let it
> Sounds reasonable. Then adding document or code comment around looks
> like a good way to go further so that people can easily get why its
> alignment is different than other reservation.
Hi Baoquan & Eric,
Sorry for late reply, i missed it earlier.
Thanks for your explanation, i will just leave the 1MB alignment here as is.
I will introduce CRASH_ALIGN_SPECIFIED to help make function
reserve_crashkernel generic.
CRASH_ALIGN_SPECIFIED is used for user specified start address which is
distinct from
default CRASH_ALIGN.
Thanks,
Chen Zhou
>
>> be the next persons problem.
>>
>> In this case there is no reason for changing this bit of code.
>> All CRASH_ALIGN is about is a default alignment when none is specified.
>> It is not a functional requirement but just something so that things
>> come out nicely.
>>
>>
>> Eric
>>
> .
>
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v14 08/11] arm64: kdump: reimplement crashkernel=X
On 2021/2/26 18:31, chenzhou wrote:
>
> On 2021/2/25 0:04, Catalin Marinas wrote:
>> On Sat, Jan 30, 2021 at 03:10:22PM +0800, Chen Zhou wrote:
>>> There are following issues in arm64 kdump:
>>> 1. We use crashkernel=X to reserve crashkernel below 4G, which
>>> will fail when there is no enough low memory.
>>> 2. If reserving crashkernel above 4G, in this case, crash dump
>>> kernel will boot failure because there is no low memory available
>>> for allocation.
>>>
>>> To solve these issues, change the behavior of crashkernel=X and
>>> introduce crashkernel=X,[high,low]. crashkernel=X tries low allocation
>>> in DMA zone, and fall back to high allocation if it fails.
>>> We can also use "crashkernel=X,high" to select a region above DMA zone,
>>> which also tries to allocate at least 256M in DMA zone automatically.
>>> "crashkernel=Y,low" can be used to allocate specified size low memory.
>>>
>>> Another minor change, there may be two regions reserved for crash
>>> dump kernel, in order to distinct from the high region and make no
>>> effect to the use of existing kexec-tools, rename the low region as
>>> "Crash kernel (low)".
>> I think we discussed this but I don't remember the conclusion. Is this
>> only renamed conditionally so that we don't break current kexec-tools?
> Yes.
>> IOW, assuming that the full crashkernel region is reserved below 4GB,
>> does the "(low)" suffix still appear or it's only if a high region is
>> additionally reserved?
> Suffix "low" only appear if a high region is additionally reserved.
>>> diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h
>>> index 3f6ecae0bc68..f0caed0cb5e1 100644
>>> --- a/arch/arm64/include/asm/kexec.h
>>> +++ b/arch/arm64/include/asm/kexec.h
>>> @@ -96,6 +96,10 @@ static inline void crash_prepare_suspend(void) {}
>>> static inline void crash_post_resume(void) {}
>>> #endif
>>>
>>> +#ifdef CONFIG_KEXEC_CORE
>>> +extern void __init reserve_crashkernel(void);
>>> +#endif
>> Why not have this in some generic header?
>>
>>> diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
>>> index c18aacde8bb0..69c592c546de 100644
>>> --- a/arch/arm64/kernel/setup.c
>>> +++ b/arch/arm64/kernel/setup.c
>>> @@ -238,7 +238,18 @@ static void __init request_standard_resources(void)
>>> kernel_data.end <= res->end)
>>> request_resource(res, &kernel_data);
>>> #ifdef CONFIG_KEXEC_CORE
>>> - /* Userspace will find "Crash kernel" region in /proc/iomem. */
>>> + /*
>>> +* Userspace will find "Crash kernel" or "Crash kernel (low)"
>>> +* region in /proc/iomem.
>>> +* In order to distinct from the high region and make no effect
>>> +* to the use of existing kexec-tools, rename the low region as
>>> +* "Crash kernel (low)".
>>> +*/
>>> + if (crashk_low_res.end && crashk_low_res.start >= res->start &&
>>> + crashk_low_res.end <= res->end) {
>>> + crashk_low_res.name = "Crash kernel (low)";
>>> + request_resource(res, &crashk_low_res);
>>> + }
>>> if (crashk_res.end && crashk_res.start >= res->start &&
>>> crashk_res.end <= res->end)
>>> request_resource(res, &crashk_res);
>> My reading of the new generic reserve_crashkernel() is that
>> crashk_low_res will only be populated if crask_res is above 4GB. If
>> that's correct, I'm fine with the renaming here since current systems
>> would not get a renamed low reservation (as long as they don't change
>> the kernel cmdline).
>>
>>> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
>>> index 912f64f505f7..d20f5c444ebf 100644
>>> --- a/arch/arm64/mm/init.c
>>> +++ b/arch/arm64/mm/init.c
>>> @@ -35,6 +35,7 @@
>>> #include
>>> #include
>>> #include
>>> +#include
>>> #include
>>> #include
>>> #include
>>> @@ -61,66 +62,11 @@ EXPORT_SYMBOL(memstart_addr);
>>> */
>>> phys_addr_t arm64_dma_phys_limit __ro_after_init;
>>
Re: [PATCH v14 08/11] arm64: kdump: reimplement crashkernel=X
On 2021/2/25 0:04, Catalin Marinas wrote:
> On Sat, Jan 30, 2021 at 03:10:22PM +0800, Chen Zhou wrote:
>> There are following issues in arm64 kdump:
>> 1. We use crashkernel=X to reserve crashkernel below 4G, which
>> will fail when there is no enough low memory.
>> 2. If reserving crashkernel above 4G, in this case, crash dump
>> kernel will boot failure because there is no low memory available
>> for allocation.
>>
>> To solve these issues, change the behavior of crashkernel=X and
>> introduce crashkernel=X,[high,low]. crashkernel=X tries low allocation
>> in DMA zone, and fall back to high allocation if it fails.
>> We can also use "crashkernel=X,high" to select a region above DMA zone,
>> which also tries to allocate at least 256M in DMA zone automatically.
>> "crashkernel=Y,low" can be used to allocate specified size low memory.
>>
>> Another minor change, there may be two regions reserved for crash
>> dump kernel, in order to distinct from the high region and make no
>> effect to the use of existing kexec-tools, rename the low region as
>> "Crash kernel (low)".
> I think we discussed this but I don't remember the conclusion. Is this
> only renamed conditionally so that we don't break current kexec-tools?
Yes.
>
> IOW, assuming that the full crashkernel region is reserved below 4GB,
> does the "(low)" suffix still appear or it's only if a high region is
> additionally reserved?
Suffix "low" only appear if a high region is additionally reserved.
>
>> diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h
>> index 3f6ecae0bc68..f0caed0cb5e1 100644
>> --- a/arch/arm64/include/asm/kexec.h
>> +++ b/arch/arm64/include/asm/kexec.h
>> @@ -96,6 +96,10 @@ static inline void crash_prepare_suspend(void) {}
>> static inline void crash_post_resume(void) {}
>> #endif
>>
>> +#ifdef CONFIG_KEXEC_CORE
>> +extern void __init reserve_crashkernel(void);
>> +#endif
> Why not have this in some generic header?
>
>> diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
>> index c18aacde8bb0..69c592c546de 100644
>> --- a/arch/arm64/kernel/setup.c
>> +++ b/arch/arm64/kernel/setup.c
>> @@ -238,7 +238,18 @@ static void __init request_standard_resources(void)
>> kernel_data.end <= res->end)
>> request_resource(res, &kernel_data);
>> #ifdef CONFIG_KEXEC_CORE
>> -/* Userspace will find "Crash kernel" region in /proc/iomem. */
>> +/*
>> + * Userspace will find "Crash kernel" or "Crash kernel (low)"
>> + * region in /proc/iomem.
>> + * In order to distinct from the high region and make no effect
>> + * to the use of existing kexec-tools, rename the low region as
>> + * "Crash kernel (low)".
>> + */
>> +if (crashk_low_res.end && crashk_low_res.start >= res->start &&
>> +crashk_low_res.end <= res->end) {
>> +crashk_low_res.name = "Crash kernel (low)";
>> +request_resource(res, &crashk_low_res);
>> +}
>> if (crashk_res.end && crashk_res.start >= res->start &&
>> crashk_res.end <= res->end)
>> request_resource(res, &crashk_res);
> My reading of the new generic reserve_crashkernel() is that
> crashk_low_res will only be populated if crask_res is above 4GB. If
> that's correct, I'm fine with the renaming here since current systems
> would not get a renamed low reservation (as long as they don't change
> the kernel cmdline).
>
>> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
>> index 912f64f505f7..d20f5c444ebf 100644
>> --- a/arch/arm64/mm/init.c
>> +++ b/arch/arm64/mm/init.c
>> @@ -35,6 +35,7 @@
>> #include
>> #include
>> #include
>> +#include
>> #include
>> #include
>> #include
>> @@ -61,66 +62,11 @@ EXPORT_SYMBOL(memstart_addr);
>> */
>> phys_addr_t arm64_dma_phys_limit __ro_after_init;
>>
>> -#ifdef CONFIG_KEXEC_CORE
>> -/*
>> - * reserve_crashkernel() - reserves memory for crash kernel
>> - *
>> - * This function reserves memory area given in "crashkernel=" kernel command
>> - * line parameter. The memory reserved is used by dump capture kernel when
>> - * primary kernel is crashing.
>> - */
>> +#ifndef CONFIG_KEXEC_CORE
>> static void __init reserve_crashkernel(void)
>> {
> [...]
>> }
>> +#endif
> Can we not have the dummy reserve_crashkernel() in the generic code as
> well and avoid the #ifndef here?
You mean put the dummy reserve_crashkernel() and the relate function
declaration in some generic header?
Baoquan also mentioned about this.
Now all the arch that support kdump have the dummy reserve_crashkernel() and
function declaration, such as arm/arm64/ppc/s390..
But currently different arch may have different CONFIG and different function
declaration about this,
for example,
for s390,
static void __init reserve_crashkernel(void)
{
#ifdef CONFIG_CRASH_DUMP
...
#endi
Re: [PATCH v14 02/11] x86: kdump: make the lower bound of crash kernel reservation consistent
On 2021/2/25 23:44, Baoquan He wrote:
> On 02/25/21 at 02:42pm, Catalin Marinas wrote:
>> On Thu, Feb 25, 2021 at 03:08:46PM +0800, Baoquan He wrote:
>>> On 02/24/21 at 02:35pm, Catalin Marinas wrote:
On Sat, Jan 30, 2021 at 03:10:16PM +0800, Chen Zhou wrote:
> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
> index da769845597d..27470479e4a3 100644
> --- a/arch/x86/kernel/setup.c
> +++ b/arch/x86/kernel/setup.c
> @@ -439,7 +439,8 @@ static int __init reserve_crashkernel_low(void)
> return 0;
> }
>
> - low_base = memblock_phys_alloc_range(low_size, CRASH_ALIGN, 0,
> CRASH_ADDR_LOW_MAX);
> + low_base = memblock_phys_alloc_range(low_size, CRASH_ALIGN, CRASH_ALIGN,
> + CRASH_ADDR_LOW_MAX);
> if (!low_base) {
> pr_err("Cannot reserve %ldMB crashkernel low memory, please try
> smaller size.\n",
> (unsigned long)(low_size >> 20));
Is there any reason why the lower bound can't be 0 in all low cases
here? (Sorry if it's been already discussed, I lost track)
>>> Seems like a good question.
>>>
>>> This reserve_crashkernel_low(), paired with reserve_crashkernel_high(), is
>>> used to reserve memory under 4G so that kdump kernel owns memory for dma
>>> buffer allocation. In that case, kernel usually is loaded in high
>>> memory. In x86_64, kernel loading need be aligned to 16M because of
>>> CONFIG_PHYSICAL_START, please see commit 32105f7fd8faa7b ("x86: find
>>> offset for crashkernel reservation automatically"). But for crashkernel
>>> low memory, there seems to be no reason to ask for 16M alignment, if
>>> it's taken as dma buffer memory.
>>>
>>> So we can make a different alignment for low memory only, e.g 2M. But
>>> 16M alignment consistent with crashkernel,high is also fine to me. The
>>> only affect is smaller alignment can increase the possibility of
>>> crashkernel low reservation.
>> I don't mind the 16M alignment in both low and high base. But is there
>> any reason that the lower bound (third argument) cannot be 0 in both
>> reserve_crashkernel() (the low attempt) and reserve_crashkernel_low()
>> cases? The comment in reserve_crashkernel() only talks about the 4G
>> upper bound but not why we need a 16M lower bound.
> Ah, sorry, I must have mixed this one with the alignment of fixed
> memory region reservation in patch 1 when considering comments.
>
> Hmm, in x86 we always have memory reserved in low 1M, lower bound
> being 0 or 16M (kernel alignment) doesn't make difference on crashkernel
> low reservation. But for crashkernel reservation, the reason should be
> kernel loading alignment being 16M, please see commit 32105f7fd8faa7b
> ("x86: find offset for crashkernel reservation automatically").
Sorry, i didn't mention in the commit message about this.
We discussed about this and the CRASH_ALIGN sounds better, so just use
CRASH_ALIGN.
https://lkml.org/lkml/2020/9/4/82
Thanks,
Chen Zhou
>
> So, for crashkernel low, keeping lower bound as 0 looks good to me, the
> only reason is just as patch log tells. And it can skip the unnecessary
> memblock searching under 16M since it will always fail, even though it
> won't matter much. Or changing it to CRASH_ALIGN as this patch is doing,
> and adding code comment, is also fine to me.
>
> Thanks
> Baoquan
>
> .
>
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v14 01/11] x86: kdump: replace the hard-coded alignment with macro CRASH_ALIGN
On 2021/2/25 15:25, Baoquan He wrote:
> On 02/24/21 at 02:19pm, Catalin Marinas wrote:
>> On Sat, Jan 30, 2021 at 03:10:15PM +0800, Chen Zhou wrote:
>>> Move CRASH_ALIGN to header asm/kexec.h for later use. Besides, the
>>> alignment of crash kernel regions in x86 is 16M(CRASH_ALIGN), but
>>> function reserve_crashkernel() also used 1M alignment. So just
>>> replace hard-coded alignment 1M with macro CRASH_ALIGN.
>> [...]
>>> @@ -510,7 +507,7 @@ static void __init reserve_crashkernel(void)
>>> } else {
>>> unsigned long long start;
>>>
>>> - start = memblock_phys_alloc_range(crash_size, SZ_1M, crash_base,
>>> + start = memblock_phys_alloc_range(crash_size, CRASH_ALIGN,
>>> crash_base,
>>> crash_base + crash_size);
>>> if (start != crash_base) {
>>> pr_info("crashkernel reservation failed - memory is in
>>> use.\n");
>> There is a small functional change here for x86. Prior to this patch,
>> crash_base passed by the user on the command line is allowed to be 1MB
>> aligned. With this patch, such reservation will fail.
>>
>> Is the current behaviour a bug in the current x86 code or it does allow
>> 1MB-aligned reservations?
> Hmm, you are right. Here we should keep 1MB alignment as is because
> users specify the address and size, their intention should be respected.
> The 1MB alignment for fixed memory region reservation was introduced in
> below commit, but it doesn't tell what is Eric's request at that time, I
> guess it meant respecting users' specifying.
I think we could make the alignment unified. Why is the alignment system
reserved and
user specified different? Besides, there is no document about the 1MB alignment.
How about adding the alignment size(16MB) in doc if user specified start
address as arm64 does.
Thanks,
Chen Zhou
>
> commit 44280733e71ad15377735b42d8538c109c94d7e3
> Author: Yinghai Lu
> Date: Sun Nov 22 17:18:49 2009 -0800
>
> x86: Change crash kernel to reserve via reserve_early()
>
> use find_e820_area()/reserve_early() instead.
>
> -v2: address Eric's request, to restore original semantics.
> will fail, if the provided address can not be used.
>
> .
>
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v14 11/11] kdump: update Documentation about crashkernel
On 2021/2/18 16:40, Baoquan He wrote: > On 01/30/21 at 03:10pm, Chen Zhou wrote: >> For arm64, the behavior of crashkernel=X has been changed, which >> tries low allocation in DMA zone and fall back to high allocation >> if it fails. >> >> We can also use "crashkernel=X,high" to select a high region above >> DMA zone, which also tries to allocate at least 256M low memory in >> DMA zone automatically and "crashkernel=Y,low" can be used to allocate >> specified size low memory. >> >> So update the Documentation. > Nice document adding which also takes care of x86 code implementation, > thanks. By the way, maybe you can remove John's 'Tested-by' since it > doesn't make much sense to test a document patch. I will remove the Tested-by in next version. > > Acked-by: Baoquan He > >> Signed-off-by: Chen Zhou >> Tested-by: John Donnelly >> --- >> Documentation/admin-guide/kdump/kdump.rst | 22 --- >> .../admin-guide/kernel-parameters.txt | 11 -- >> 2 files changed, 28 insertions(+), 5 deletions(-) >> >> diff --git a/Documentation/admin-guide/kdump/kdump.rst >> b/Documentation/admin-guide/kdump/kdump.rst >> index 75a9dd98e76e..0877c76f8015 100644 >> --- a/Documentation/admin-guide/kdump/kdump.rst >> +++ b/Documentation/admin-guide/kdump/kdump.rst >> @@ -299,7 +299,16 @@ Boot into System Kernel >> "crashkernel=64M@16M" tells the system kernel to reserve 64 MB of memory >> starting at physical address 0x0100 (16MB) for the dump-capture >> kernel. >> >> - On x86 and x86_64, use "crashkernel=64M@16M". >> + On x86 use "crashkernel=64M@16M". >> + >> + On x86_64, use "crashkernel=X" to select a region under 4G first, and >> + fall back to reserve region above 4G. And go for high allocation >> + directly if the required size is too large. >> + We can also use "crashkernel=X,high" to select a region above 4G, which >> + also tries to allocate at least 256M below 4G automatically and >> + "crashkernel=Y,low" can be used to allocate specified size low memory. >> + Use "crashkernel=Y@X" if you really have to reserve memory from specified >> + start address X. >> >> On ppc64, use "crashkernel=128M@32M". >> >> @@ -316,8 +325,15 @@ Boot into System Kernel >> kernel will automatically locate the crash kernel image within the >> first 512MB of RAM if X is not given. >> >> - On arm64, use "crashkernel=Y[@X]". Note that the start address of >> - the kernel, X if explicitly specified, must be aligned to 2MiB >> (0x20). >> + On arm64, use "crashkernel=X" to try low allocation in DMA zone and >> + fall back to high allocation if it fails. >> + We can also use "crashkernel=X,high" to select a high region above >> + DMA zone, which also tries to allocate at least 256M low memory in >> + DMA zone automatically. >> + "crashkernel=Y,low" can be used to allocate specified size low memory. >> + Use "crashkernel=Y@X" if you really have to reserve memory from >> + specified start address X. Note that the start address of the kernel, >> + X if explicitly specified, must be aligned to 2MiB (0x20). >> >> Load the Dump-capture Kernel >> >> diff --git a/Documentation/admin-guide/kernel-parameters.txt >> b/Documentation/admin-guide/kernel-parameters.txt >> index a10b545c2070..908e5c8b61ba 100644 >> --- a/Documentation/admin-guide/kernel-parameters.txt >> +++ b/Documentation/admin-guide/kernel-parameters.txt >> @@ -738,6 +738,9 @@ >> [KNL, X86-64] Select a region under 4G first, and >> fall back to reserve region above 4G when '@offset' >> hasn't been specified. >> +[KNL, arm64] Try low allocation in DMA zone and fall >> back >> +to high allocation if it fails when '@offset' hasn't >> been >> +specified. >> See Documentation/admin-guide/kdump/kdump.rst for >> further details. >> >> crashkernel=range1:size1[,range2:size2,...][@offset] >> @@ -754,6 +757,8 @@ >> Otherwise memory region will be allocated below 4G, if >> available. >> It will be ignored if crashkernel=X is specified. >> +[KNL, arm64] range in high memory. >> +Allow kernel to allocate physical memory region from >> top. >> crashkernel=size[KMG],low >> [KNL, X86-64] range under 4G. When crashkernel=X,high >> is passed, kernel could allocate physical memory region >> @@ -762,13 +767,15 @@ >> requires at least 64M+32K low memory, also enough extra >> low memory is needed to make sure DMA buffers for 32-bit >> devices won't run out. Kernel would try to allocate at >> -at least 256M below 4G automatically. >> +least 256M below 4G automa
Re: [PATCH v14 09/11] x86, arm64: Add ARCH_WANT_RESERVE_CRASH_KERNEL config
On 2021/2/18 16:35, Baoquan He wrote:
> On 01/30/21 at 03:10pm, Chen Zhou wrote:
>> We make the functions reserve_crashkernel[_low]() as generic for
>> x86 and arm64. Since reserve_crashkernel[_low]() implementations
>> are quite similar on other architectures as well, we can have more
>> users of this later.
>>
>> So have CONFIG_ARCH_WANT_RESERVE_CRASH_KERNEL in arch/Kconfig and
>> select this by X86 and ARM64.
> This looks much better with the help of
> CONFIG_ARCH_WANT_RESERVE_CRASH_KERNEL. And please take off the
> 'Suggested-by' tag from me, I just don't like the old CONFIG_X86 and
> CONFIG_ARM64 ifdeffery way in v13, Mike suggested this ARCH_WANT_
> option.
OK, i will delete this.
>
> And the two dummy function reserve_crashkernel() in x86 and arm64 looks
> not so good, but I don't have better idea. Maybe add
> CONFIG_ARCH_WANT_RESERVE_CRASH_KERNEL iddeffery in the call site of
> reserve_crashkernel() in each ARCH? Or just leave with it for now if no
> other people has concern or suggestion about it.
>
> Anyway, ack this one.
>
> Acked-by: Baoquan He
>
> Thanks
> Baoquan
>
>
>> Suggested-by: Mike Rapoport
>> Suggested-by: Baoquan He
>> Signed-off-by: Chen Zhou
>> ---
>> arch/Kconfig| 3 +++
>> arch/arm64/Kconfig | 1 +
>> arch/x86/Kconfig| 2 ++
>> kernel/crash_core.c | 7 ++-
>> 4 files changed, 8 insertions(+), 5 deletions(-)
>>
>> diff --git a/arch/Kconfig b/arch/Kconfig
>> index 24862d15f3a3..0ca1ff5bb157 100644
>> --- a/arch/Kconfig
>> +++ b/arch/Kconfig
>> @@ -24,6 +24,9 @@ config KEXEC_ELF
>> config HAVE_IMA_KEXEC
>> bool
>>
>> +config ARCH_WANT_RESERVE_CRASH_KERNEL
>> +bool
>> +
>> config SET_FS
>> bool
>>
>> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
>> index f39568b28ec1..09365c7ff469 100644
>> --- a/arch/arm64/Kconfig
>> +++ b/arch/arm64/Kconfig
>> @@ -82,6 +82,7 @@ config ARM64
>> select ARCH_WANT_FRAME_POINTERS
>> select ARCH_WANT_HUGE_PMD_SHARE if ARM64_4K_PAGES || (ARM64_16K_PAGES
>> && !ARM64_VA_BITS_36)
>> select ARCH_WANT_LD_ORPHAN_WARN
>> +select ARCH_WANT_RESERVE_CRASH_KERNEL if KEXEC_CORE
>> select ARCH_HAS_UBSAN_SANITIZE_ALL
>> select ARM_AMBA
>> select ARM_ARCH_TIMER
>> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
>> index 21f851179ff0..e6926fcb4a40 100644
>> --- a/arch/x86/Kconfig
>> +++ b/arch/x86/Kconfig
>> @@ -12,6 +12,7 @@ config X86_32
>> depends on !64BIT
>> # Options that are inherently 32-bit kernel only:
>> select ARCH_WANT_IPC_PARSE_VERSION
>> +select ARCH_WANT_RESERVE_CRASH_KERNEL if KEXEC_CORE
>> select CLKSRC_I8253
>> select CLONE_BACKWARDS
>> select GENERIC_VDSO_32
>> @@ -28,6 +29,7 @@ config X86_64
>> select ARCH_HAS_GIGANTIC_PAGE
>> select ARCH_SUPPORTS_INT128 if CC_HAS_INT128
>> select ARCH_USE_CMPXCHG_LOCKREF
>> +select ARCH_WANT_RESERVE_CRASH_KERNEL if KEXEC_CORE
>> select HAVE_ARCH_SOFT_DIRTY
>> select MODULES_USE_ELF_RELA
>> select NEED_DMA_MAP_STATE
>> diff --git a/kernel/crash_core.c b/kernel/crash_core.c
>> index 8479be270c0b..2c5783985db5 100644
>> --- a/kernel/crash_core.c
>> +++ b/kernel/crash_core.c
>> @@ -320,9 +320,7 @@ int __init parse_crashkernel_low(char *cmdline,
>> * - Crashkernel reservation --
>> */
>>
>> -#ifdef CONFIG_KEXEC_CORE
>> -
>> -#if defined(CONFIG_X86) || defined(CONFIG_ARM64)
>> +#ifdef CONFIG_ARCH_WANT_RESERVE_CRASH_KERNEL
>> static int __init reserve_crashkernel_low(void)
>> {
>> #ifdef CONFIG_64BIT
>> @@ -450,8 +448,7 @@ void __init reserve_crashkernel(void)
>> crashk_res.start = crash_base;
>> crashk_res.end = crash_base + crash_size - 1;
>> }
>> -#endif
>> -#endif /* CONFIG_KEXEC_CORE */
>> +#endif /* CONFIG_ARCH_WANT_RESERVE_CRASH_KERNEL */
>>
>> Elf_Word *append_elf_note(Elf_Word *buf, char *name, unsigned int type,
>>void *data, size_t data_len)
>> --
>> 2.20.1
>>
> .
>
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v14 06/11] x86/elf: Move vmcore_elf_check_arch_cross to arch/x86/include/asm/elf.h
On 2021/2/18 14:31, Baoquan He wrote:
> On 01/30/21 at 03:10pm, Chen Zhou wrote:
>> Move macro vmcore_elf_check_arch_cross from arch/x86/include/asm/kexec.h
>> to arch/x86/include/asm/elf.h to fix the following compiling warning:
>>
>> make ARCH=i386
>> In file included from arch/x86/kernel/setup.c:39:0:
>> ./arch/x86/include/asm/kexec.h:77:0: warning: "vmcore_elf_check_arch_cross"
>> redefined
>> # define vmcore_elf_check_arch_cross(x) ((x)->e_machine == EM_X86_64)
>>
>> In file included from arch/x86/kernel/setup.c:9:0:
>> ./include/linux/crash_dump.h:39:0: note: this is the location of the
>> previous definition
>> #define vmcore_elf_check_arch_cross(x) 0
>>
>> The root cause is that vmcore_elf_check_arch_cross under CONFIG_CRASH_CORE
>> depend on CONFIG_KEXEC_CORE. Commit 2db65f1db17d ("x86: kdump: move
>> reserve_crashkernel[_low]() into crash_core.c") triggered the issue.
>>
>> Suggested by Mike, simply move vmcore_elf_check_arch_cross from
>> arch/x86/include/asm/kexec.h to arch/x86/include/asm/elf.h to fix
>> the warning.
>>
>> Fixes: 2db65f1db17d ("x86: kdump: move reserve_crashkernel[_low]() into
>> crash_core.c")
> Where does this commit id '2db65f1db17d' come from? Here you are fixing
> another pathc in the same patchset. Please merge this with patch 05/11.
Yeah, the commit id is invalid, i will merge this patch with patch 05/11.
Thanks,
Chen Zhou
>
>> Reported-by: kernel test robot
>> Suggested-by: Mike Rapoport
>> Signed-off-by: Chen Zhou
>> ---
>> arch/x86/include/asm/elf.h | 3 +++
>> arch/x86/include/asm/kexec.h | 3 ---
>> 2 files changed, 3 insertions(+), 3 deletions(-)
>>
>> diff --git a/arch/x86/include/asm/elf.h b/arch/x86/include/asm/elf.h
>> index 66bdfe838d61..5333777cc758 100644
>> --- a/arch/x86/include/asm/elf.h
>> +++ b/arch/x86/include/asm/elf.h
>> @@ -94,6 +94,9 @@ extern unsigned int vdso32_enabled;
>>
>> #define elf_check_arch(x) elf_check_arch_ia32(x)
>>
>> +/* We can also handle crash dumps from 64 bit kernel. */
>> +# define vmcore_elf_check_arch_cross(x) ((x)->e_machine == EM_X86_64)
>> +
>> /* SVR4/i386 ABI (pages 3-31, 3-32) says that when the program starts %edx
>> contains a pointer to a function which might be registered using
>> `atexit'.
>> This provides a mean for the dynamic linker to call DT_FINI functions for
>> diff --git a/arch/x86/include/asm/kexec.h b/arch/x86/include/asm/kexec.h
>> index 2b18f918203e..6fcae01a9cca 100644
>> --- a/arch/x86/include/asm/kexec.h
>> +++ b/arch/x86/include/asm/kexec.h
>> @@ -72,9 +72,6 @@ struct kimage;
>>
>> /* The native architecture */
>> # define KEXEC_ARCH KEXEC_ARCH_386
>> -
>> -/* We can also handle crash dumps from 64 bit kernel. */
>> -# define vmcore_elf_check_arch_cross(x) ((x)->e_machine == EM_X86_64)
>> #else
>> /* Maximum physical address we can use pages from */
>> # define KEXEC_SOURCE_MEMORY_LIMIT (MAXMEM-1)
>> --
>> 2.20.1
>>
> .
>
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v14 00/11] support reserving crashkernel above 4G on arm64 kdump
Hi all, Friendly ping... On 2021/1/30 15:10, Chen Zhou wrote: > There are following issues in arm64 kdump: > 1. We use crashkernel=X to reserve crashkernel below 4G, which > will fail when there is no enough low memory. > 2. If reserving crashkernel above 4G, in this case, crash dump > kernel will boot failure because there is no low memory available > for allocation. > > To solve these issues, change the behavior of crashkernel=X. > crashkernel=X tries low allocation in DMA zone and fall back to high > allocation if it fails. > > We can also use "crashkernel=X,high" to select a high region above > DMA zone, which also tries to allocate at least 256M low memory in > DMA zone automatically and "crashkernel=Y,low" can be used to allocate > specified size low memory. > > When reserving crashkernel in high memory, some low memory is reserved > for crash dump kernel devices. So there may be two regions reserved for > crash dump kernel. > In order to distinct from the high region and make no effect to the use > of existing kexec-tools, rename the low region as "Crash kernel (low)", > and pass the low region by reusing DT property > "linux,usable-memory-range". We made the low memory region as the last > range of "linux,usable-memory-range" to keep compatibility with existing > user-space and older kdump kernels. > > Besides, we need to modify kexec-tools: > arm64: support more than one crash kernel regions(see [1]) > > Another update is document about DT property 'linux,usable-memory-range': > schemas: update 'linux,usable-memory-range' node schema(see [2]) > > This patchset contains the following eleven patches: > 0001-x86-kdump-replace-the-hard-coded-alignment-with-macr.patch > 0002-x86-kdump-make-the-lower-bound-of-crash-kernel-reser.patch > 0003-x86-kdump-use-macro-CRASH_ADDR_LOW_MAX-in-functions-.patch > 0004-x86-kdump-move-xen_pv_domain-check-and-insert_resour.patch > 0005-x86-kdump-move-reserve_crashkernel-_low-into-crash_c.patch > 0006-x86-elf-Move-vmcore_elf_check_arch_cross-to-arch-x86.patch > 0007-arm64-kdump-introduce-some-macroes-for-crash-kernel-.patch > 0008-arm64-kdump-reimplement-crashkernel-X.patch > 0009-x86-arm64-Add-ARCH_WANT_RESERVE_CRASH_KERNEL-config.patch > 0010-arm64-kdump-add-memory-for-devices-by-DT-property-li.patch > 0011-kdump-update-Documentation-about-crashkernel.patch > > 0001-0004 are some x86 cleanups which prepares for making > functionsreserve_crashkernel[_low]() generic. > 0005 makes functions reserve_crashkernel[_low]() generic. > 0006 fix compiling warning. > 0007-0009 reimplements arm64 crashkernel=X. > 0010 adds memory for devices by DT property linux,usable-memory-range. > 0011 updates the doc. > > Changes since [v13] > - Rebased on top of 5.11-rc5. > - Introduce config CONFIG_ARCH_WANT_RESERVE_CRASH_KERNEL. > Since reserve_crashkernel[_low]() implementations are quite similar on > other architectures, so have CONFIG_ARCH_WANT_RESERVE_CRASH_KERNEL in > arch/Kconfig and select this by X86 and ARM64. > - Some minor cleanup. > > Changes since [v12] > - Rebased on top of 5.10-rc1. > - Keep CRASH_ALIGN as 16M suggested by Dave. > - Drop patch "kdump: add threshold for the required memory". > - Add Tested-by from John. > > Changes since [v11] > - Rebased on top of 5.9-rc4. > - Make the function reserve_crashkernel() of x86 generic. > Suggested by Catalin, make the function reserve_crashkernel() of x86 generic > and arm64 use the generic version to reimplement crashkernel=X. > > Changes since [v10] > - Reimplement crashkernel=X suggested by Catalin, Many thanks to Catalin. > > Changes since [v9] > - Patch 1 add Acked-by from Dave. > - Update patch 5 according to Dave's comments. > - Update chosen schema. > > Changes since [v8] > - Reuse DT property "linux,usable-memory-range". > Suggested by Rob, reuse DT property "linux,usable-memory-range" to pass the > low > memory region. > - Fix kdump broken with ZONE_DMA reintroduced. > - Update chosen schema. > > Changes since [v7] > - Move x86 CRASH_ALIGN to 2M > Suggested by Dave and do some test, move x86 CRASH_ALIGN to 2M. > - Update Documentation/devicetree/bindings/chosen.txt. > Add corresponding documentation to > Documentation/devicetree/bindings/chosen.txt > suggested by Arnd. > - Add Tested-by from Jhon and pk. > > Changes since [v6] > - Fix build errors reported by kbuild test robot. > > Changes since [v5] > - Move reserve_crashkernel_low() into kernel/crash_core.c. > - Delete crashkernel=X,high. > - Modify crashkernel=X,low. > If crashkernel=X,low is specified simultaneously, reserve spcified size low > memory for crash kdump kernel devices firstly and then reserve memory above > 4G. > In addition, rename crashk_low_res as "Crash kernel (low)" for arm64, and then > pass to crash dump kernel by DT property "linux,low-memory-range". > - Update Documentation/admin-guide/kdump/kdump.rst. > > Changes since [v4] > - Reimplement memblock_cap_memory_ranges for multiple ranges by Mike. > > Changes since [v3] > - Add memblock_c
Re: [PATCH v14 11/11] kdump: update Documentation about crashkernel
Hi Randy, On 2021/1/31 1:53, Randy Dunlap wrote: > Hi-- > > On 1/29/21 11:10 PM, Chen Zhou wrote: >> --- >> Documentation/admin-guide/kdump/kdump.rst | 22 --- >> .../admin-guide/kernel-parameters.txt | 11 -- >> 2 files changed, 28 insertions(+), 5 deletions(-) >> >> diff --git a/Documentation/admin-guide/kernel-parameters.txt >> b/Documentation/admin-guide/kernel-parameters.txt >> index a10b545c2070..908e5c8b61ba 100644 >> --- a/Documentation/admin-guide/kernel-parameters.txt >> +++ b/Documentation/admin-guide/kernel-parameters.txt > All of the "arm64" instances in [square brackets] should be "ARM64". Got it, thanks for your review. Thanks, Chen Zhou > >> @@ -738,6 +738,9 @@ >> [KNL, X86-64] Select a region under 4G first, and >> fall back to reserve region above 4G when '@offset' >> hasn't been specified. >> +[KNL, arm64] Try low allocation in DMA zone and fall >> back > here > >> +to high allocation if it fails when '@offset' hasn't >> been >> +specified. >> See Documentation/admin-guide/kdump/kdump.rst for >> further details. >> >> crashkernel=range1:size1[,range2:size2,...][@offset] >> @@ -754,6 +757,8 @@ >> Otherwise memory region will be allocated below 4G, if >> available. >> It will be ignored if crashkernel=X is specified. >> +[KNL, arm64] range in high memory. > here > >> +Allow kernel to allocate physical memory region from >> top. >> crashkernel=size[KMG],low >> [KNL, X86-64] range under 4G. When crashkernel=X,high >> is passed, kernel could allocate physical memory region >> @@ -762,13 +767,15 @@ >> requires at least 64M+32K low memory, also enough extra >> low memory is needed to make sure DMA buffers for 32-bit >> devices won't run out. Kernel would try to allocate at >> -at least 256M below 4G automatically. >> +least 256M below 4G automatically. >> This one let user to specify own low range under 4G >> for second kernel instead. >> 0: to disable low allocation. >> It will be ignored when crashkernel=X,high is not used >> or memory reserved is below 4G. >> - >> +[KNL, arm64] range in low memory. > here > >> +This one let user to specify a low range in DMA zone for >> +crash dump kernel. > > Thanks. > ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v13 6/8] arm64: kdump: reimplement crashkernel=X
On 2020/11/12 16:36, Baoquan He wrote:
> On 11/12/20 at 10:25am, Mike Rapoport wrote:
>> On Wed, Nov 11, 2020 at 09:54:48PM +0800, Baoquan He wrote:
>>> On 11/11/20 at 09:27pm, chenzhou wrote:
>>>> Hi Baoquan,
>>> ...
>>>>>> #ifdef CONFIG_CRASH_DUMP
>>>>>> static int __init early_init_dt_scan_elfcorehdr(unsigned long node,
>>>>>> diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
>>>>>> index 1c0f3e02f731..c55cee290bbb 100644
>>>>>> --- a/arch/arm64/mm/mmu.c
>>>>>> +++ b/arch/arm64/mm/mmu.c
>>>>>> @@ -488,6 +488,10 @@ static void __init map_mem(pgd_t *pgdp)
>>>>>> */
>>>>>> memblock_mark_nomap(kernel_start, kernel_end - kernel_start);
>>>>>> #ifdef CONFIG_KEXEC_CORE
>>>>>> +if (crashk_low_res.end)
>>>>>> +memblock_mark_nomap(crashk_low_res.start,
>>>>>> +resource_size(&crashk_low_res));
>>>>>> +
>>>>>> if (crashk_res.end)
>>>>>> memblock_mark_nomap(crashk_res.start,
>>>>>> resource_size(&crashk_res));
>>>>>> diff --git a/kernel/crash_core.c b/kernel/crash_core.c
>>>>>> index d39892bdb9ae..cdef7d8c91a6 100644
>>>>>> --- a/kernel/crash_core.c
>>>>>> +++ b/kernel/crash_core.c
>>>>>> @@ -321,7 +321,7 @@ int __init parse_crashkernel_low(char *cmdline,
>>>>>>
>>>>>> int __init reserve_crashkernel_low(void)
>>>>>> {
>>>>>> -#ifdef CONFIG_X86_64
>>>>>> +#if defined(CONFIG_X86_64) || defined(CONFIG_ARM64)
>>>>> Not very sure if a CONFIG_64BIT checking is better.
>>>> If doing like this, there may be some compiling errors for other 64-bit
>>>> kernel, such as mips.
>>>>>> unsigned long long base, low_base = 0, low_size = 0;
>>>>>> unsigned long low_mem_limit;
>>>>>> int ret;
>>>>>> @@ -362,12 +362,14 @@ int __init reserve_crashkernel_low(void)
>>>>>>
>>>>>> crashk_low_res.start = low_base;
>>>>>> crashk_low_res.end = low_base + low_size - 1;
>>>>>> +#ifdef CONFIG_X86_64
>>>>>> insert_resource(&iomem_resource, &crashk_low_res);
>>>>>> +#endif
>>>>>> #endif
>>>>>> return 0;
>>>>>> }
>>>>>>
>>>>>> -#ifdef CONFIG_X86
>>>>>> +#if defined(CONFIG_X86) || defined(CONFIG_ARM64)
>>>>> Should we make this weak default so that we can remove the ARCH config?
>>>> The same as above, some arch may not support kdump, in that case,
>>>> compiling errors occur.
>>> OK, not sure if other people have better idea, oterwise, we can leave with
>>> it.
>>> Thanks for telling.
>> I think it would be better to have CONFIG_ARCH_WANT_RESERVE_CRASH_KERNEL
>> in arch/Kconfig and select this by X86 and ARM64.
>>
>> Since reserve_crashkernel() implementations are quite similart on other
>> architectures as well, we can have more users of this later.
> Yes, this sounds like a nice way.
I will think about this in next version.
Thanks,
Chen Zhou
>
> .
>
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v13 4/8] x86: kdump: move reserve_crashkernel[_low]() into crash_core.c
On 2020/11/12 16:11, Mike Rapoport wrote:
> On Sat, Oct 31, 2020 at 03:44:33PM +0800, Chen Zhou wrote:
>> Make the functions reserve_crashkernel[_low]() as generic.
>> Arm64 will use these to reimplement crashkernel=X.
>>
>> Signed-off-by: Chen Zhou
>> Tested-by: John Donnelly
>> ---
>> arch/x86/include/asm/kexec.h | 25 ++
>> arch/x86/kernel/setup.c | 151 +---
>> include/linux/crash_core.h | 4 +
>> include/linux/kexec.h| 2 -
>> kernel/crash_core.c | 164 +++
>> kernel/kexec_core.c | 17
>> 6 files changed, 195 insertions(+), 168 deletions(-)
>>
>> diff --git a/arch/x86/include/asm/kexec.h b/arch/x86/include/asm/kexec.h
>> index 8cf9d3fd31c7..34afa7b645f9 100644
>> --- a/arch/x86/include/asm/kexec.h
>> +++ b/arch/x86/include/asm/kexec.h
>> @@ -21,6 +21,27 @@
>> /* 2M alignment for crash kernel regions */
>> #define CRASH_ALIGN SZ_16M
>>
>> +/*
>> + * Keep the crash kernel below this limit.
>> + *
>> + * Earlier 32-bits kernels would limit the kernel to the low 512 MB range
>> + * due to mapping restrictions.
>> + *
>> + * 64-bit kdump kernels need to be restricted to be under 64 TB, which is
>> + * the upper limit of system RAM in 4-level paging mode. Since the kdump
>> + * jump could be from 5-level paging to 4-level paging, the jump will fail
>> if
>> + * the kernel is put above 64 TB, and during the 1st kernel bootup there's
>> + * no good way to detect the paging mode of the target kernel which will be
>> + * loaded for dumping.
>> + */
>> +#ifdef CONFIG_X86_32
>> +# define CRASH_ADDR_LOW_MAX SZ_512M
>> +# define CRASH_ADDR_HIGH_MAXSZ_512M
>> +#else
>> +# define CRASH_ADDR_LOW_MAX SZ_4G
>> +# define CRASH_ADDR_HIGH_MAXSZ_64T
>> +#endif
>> +
>> #ifndef __ASSEMBLY__
>>
>> #include
>> @@ -200,6 +221,10 @@ typedef void crash_vmclear_fn(void);
>> extern crash_vmclear_fn __rcu *crash_vmclear_loaded_vmcss;
>> extern void kdump_nmi_shootdown_cpus(void);
>>
>> +#ifdef CONFIG_KEXEC_CORE
>> +extern void __init reserve_crashkernel(void);
>> +#endif
>> +
>> #endif /* __ASSEMBLY__ */
>>
>> #endif /* _ASM_X86_KEXEC_H */
>> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
>> index 1289f079ad5f..00b3840d30f9 100644
>> --- a/arch/x86/kernel/setup.c
>> +++ b/arch/x86/kernel/setup.c
>> @@ -25,8 +25,6 @@
>>
>> #include
>>
>> -#include
>> -
>> #include
>> #include
>> #include
>> @@ -38,6 +36,7 @@
>> #include
>> #include
>> #include
>> +#include
>> #include
>> #include
>> #include
>> @@ -389,153 +388,7 @@ static void __init
>> memblock_x86_reserve_range_setup_data(void)
>> }
>> }
>>
>> -/*
>> - * - Crashkernel reservation --
>> - */
>> -
>> -#ifdef CONFIG_KEXEC_CORE
>> -
>> -/*
>> - * Keep the crash kernel below this limit.
>> - *
>> - * Earlier 32-bits kernels would limit the kernel to the low 512 MB range
>> - * due to mapping restrictions.
>> - *
>> - * 64-bit kdump kernels need to be restricted to be under 64 TB, which is
>> - * the upper limit of system RAM in 4-level paging mode. Since the kdump
>> - * jump could be from 5-level paging to 4-level paging, the jump will fail
>> if
>> - * the kernel is put above 64 TB, and during the 1st kernel bootup there's
>> - * no good way to detect the paging mode of the target kernel which will be
>> - * loaded for dumping.
>> - */
>> -#ifdef CONFIG_X86_32
>> -# define CRASH_ADDR_LOW_MAX SZ_512M
>> -# define CRASH_ADDR_HIGH_MAXSZ_512M
>> -#else
>> -# define CRASH_ADDR_LOW_MAX SZ_4G
>> -# define CRASH_ADDR_HIGH_MAXSZ_64T
>> -#endif
>> -
>> -static int __init reserve_crashkernel_low(void)
>> -{
>> -#ifdef CONFIG_X86_64
>> -unsigned long long base, low_base = 0, low_size = 0;
>> -unsigned long low_mem_limit;
>> -int ret;
>> -
>> -low_mem_limit = min(memblock_phys_mem_size(), CRASH_ADDR_LOW_MAX);
>> -
>> -/* crashkernel=Y,low */
>> -ret = parse_crashkernel_low(boot_command_line, low_mem_limit,
>> &low_size, &base);
>> -if (ret) {
>> -/*
>> - * two parts from kernel/dma/swiotlb.c:
>> - * -swiotlb size: user-specified with swiotlb= or default.
>> - *
>> - * -swiotlb overflow buffer: now hardcoded to 32k. We round it
>> - * to 8M for other buffers that may need to stay low too. Also
>> - * make sure we allocate enough extra low memory so that we
>> - * don't run out of DMA buffers for 32-bit devices.
>> - */
>> -low_size = max(swiotlb_size_or_default() + (8UL << 20), 256UL
>> << 20);
>> -} else {
>> -/* passed with crashkernel=0,low ? */
>> -if (!low_size)
>> -return 0;
>> -}
>> -
>> -low_base = memblock_phys_alloc_range(low_size, CRASH_ALIGN,
>> CRASH_ALIGN, CRASH_ADDR_LOW_MAX);
>> -if (!low_base) {
>> -pr_err("C
Re: [PATCH v13 1/8] x86: kdump: replace the hard-coded alignment with macro CRASH_ALIGN
On 2020/11/12 15:58, Mike Rapoport wrote:
> Hi,
>
> On Sat, Oct 31, 2020 at 03:44:30PM +0800, Chen Zhou wrote:
>> Move CRASH_ALIGN to header asm/kexec.h and replace the hard-coded
>> alignment with macro CRASH_ALIGN in function reserve_crashkernel().
>>
>> Suggested-by: Dave Young
>> Signed-off-by: Chen Zhou
>> Tested-by: John Donnelly
>> ---
>> arch/x86/include/asm/kexec.h | 3 +++
>> arch/x86/kernel/setup.c | 5 +
>> 2 files changed, 4 insertions(+), 4 deletions(-)
>>
>> diff --git a/arch/x86/include/asm/kexec.h b/arch/x86/include/asm/kexec.h
>> index 6802c59e8252..8cf9d3fd31c7 100644
>> --- a/arch/x86/include/asm/kexec.h
>> +++ b/arch/x86/include/asm/kexec.h
>> @@ -18,6 +18,9 @@
>>
>> # define KEXEC_CONTROL_CODE_MAX_SIZE2048
>>
>> +/* 2M alignment for crash kernel regions */
>> +#define CRASH_ALIGN SZ_16M
> Please update the comment to match the code.
Ok, thanks for pointing this mistake.
Thanks,
Chen Zhou
>
>> +
>> #ifndef __ASSEMBLY__
>>
>> #include
>> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
>> index 84f581c91db4..bf373422dc8a 100644
>> --- a/arch/x86/kernel/setup.c
>> +++ b/arch/x86/kernel/setup.c
>> @@ -395,9 +395,6 @@ static void __init
>> memblock_x86_reserve_range_setup_data(void)
>>
>> #ifdef CONFIG_KEXEC_CORE
>>
>> -/* 16M alignment for crash kernel regions */
>> -#define CRASH_ALIGN SZ_16M
>> -
>> /*
>> * Keep the crash kernel below this limit.
>> *
>> @@ -515,7 +512,7 @@ static void __init reserve_crashkernel(void)
>> } else {
>> unsigned long long start;
>>
>> -start = memblock_phys_alloc_range(crash_size, SZ_1M, crash_base,
>> +start = memblock_phys_alloc_range(crash_size, CRASH_ALIGN,
>> crash_base,
>>crash_base + crash_size);
>> if (start != crash_base) {
>> pr_info("crashkernel reservation failed - memory is in
>> use.\n");
>> --
>> 2.20.1
>>
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v13 0/8] support reserving crashkernel above 4G on arm64 kdump
Hi Baoquan, Bhupesh,
On 2020/11/11 11:01, Baoquan He wrote:
> Hi Zhou, Bhupesh
>
> On 10/31/20 at 03:44pm, Chen Zhou wrote:
>> There are following issues in arm64 kdump:
>> 1. We use crashkernel=X to reserve crashkernel below 4G, which
>> will fail when there is no enough low memory.
>> 2. If reserving crashkernel above 4G, in this case, crash dump
>> kernel will boot failure because there is no low memory available
>> for allocation.
>> 3. Since commit 1a8e1cef7603 ("arm64: use both ZONE_DMA and ZONE_DMA32"),
>> if the memory reserved for crash dump kernel falled in ZONE_DMA32,
>> the devices in crash dump kernel need to use ZONE_DMA will alloc
>> fail.
> I went through this patchset, mainly the x86 related and generic
> changes, the changes look great and no risk. And I know Bhupesh is
> following up this and helping review, thanks, both.
>
> So you have also tested crashkernel reservation on x86_64, with the
> normal reservation, and high/low reservation, it is working well,
> right? Asking this because I didn't see the test result description, and
> just note it.
Yeah, i also tested on x86_64 and work well. I did these basic tests before
sending every
new version.
But Bhupesh may have some review comments(Bhupesh referred one month ago).
Thanks,
Chen Zhou
>
> Thanks
> Baoquan
>
>> To solve these issues, change the behavior of crashkernel=X.
>> crashkernel=X tries low allocation in DMA zone (or the DMA32 zone if
>> CONFIG_ZONE_DMA is disabled), and fall back to high allocation if it fails.
>>
>> We can also use "crashkernel=X,high" to select a high region above
>> DMA zone, which also tries to allocate at least 256M low memory in
>> DMA zone automatically (or the DMA32 zone if CONFIG_ZONE_DMA is disabled).
>> "crashkernel=Y,low" can be used to allocate specified size low memory.
>>
>> When reserving crashkernel in high memory, some low memory is reserved
>> for crash dump kernel devices. So there may be two regions reserved for
>> crash dump kernel.
>> In order to distinct from the high region and make no effect to the use
>> of existing kexec-tools, rename the low region as "Crash kernel (low)",
>> and pass the low region by reusing DT property
>> "linux,usable-memory-range". We made the low memory region as the last
>> range of "linux,usable-memory-range" to keep compatibility with existing
>> user-space and older kdump kernels.
>>
>> Besides, we need to modify kexec-tools:
>> arm64: support more than one crash kernel regions(see [1])
>>
>> Another update is document about DT property 'linux,usable-memory-range':
>> schemas: update 'linux,usable-memory-range' node schema(see [2])
>>
>> This patchset contains the following eight patches:
>> 0001-x86-kdump-replace-the-hard-coded-alignment-with-macr.patch
>> 0002-x86-kdump-make-the-lower-bound-of-crash-kernel-reser.patch
>> 0003-x86-kdump-use-macro-CRASH_ADDR_LOW_MAX-in-functions-.patch
>> 0004-x86-kdump-move-reserve_crashkernel-_low-into-crash_c.patch
>> 0005-arm64-kdump-introduce-some-macroes-for-crash-kernel-.patch
>> 0006-arm64-kdump-reimplement-crashkernel-X.patch
>> 0007-arm64-kdump-add-memory-for-devices-by-DT-property-li.patch
>> 0008-kdump-update-Documentation-about-crashkernel.patch
>>
>> 0001-0003 are some x86 cleanups which prepares for making
>> functionsreserve_crashkernel[_low]() generic.
>> 0004 makes functions reserve_crashkernel[_low]() generic.
>> 0005-0006 reimplements arm64 crashkernel=X.
>> 0007 adds memory for devices by DT property linux,usable-memory-range.
>> 0008 updates the doc.
>>
>> Changes since [v12]
>> - Rebased on top of 5.10-rc1.
>> - Keep CRASH_ALIGN as 16M suggested by Dave.
>> - Drop patch "kdump: add threshold for the required memory".
>> - Add Tested-by from John.
>>
>> Changes since [v11]
>> - Rebased on top of 5.9-rc4.
>> - Make the function reserve_crashkernel() of x86 generic.
>> Suggested by Catalin, make the function reserve_crashkernel() of x86 generic
>> and arm64 use the generic version to reimplement crashkernel=X.
>>
>> Changes since [v10]
>> - Reimplement crashkernel=X suggested by Catalin, Many thanks to Catalin.
>>
>> Changes since [v9]
>> - Patch 1 add Acked-by from Dave.
>> - Update patch 5 according to Dave's comments.
>> - Update chosen schema.
>>
>> Changes since [v8]
>> - Reuse DT property "linux,usable-memory-range".
>> Suggested by Rob, reuse DT property "linux,usable-memory-range" to pass the
>> low
>> memory region.
>> - Fix kdump broken with ZONE_DMA reintroduced.
>> - Update chosen schema.
>>
>> Changes since [v7]
>> - Move x86 CRASH_ALIGN to 2M
>> Suggested by Dave and do some test, move x86 CRASH_ALIGN to 2M.
>> - Update Documentation/devicetree/bindings/chosen.txt.
>> Add corresponding documentation to
>> Documentation/devicetree/bindings/chosen.txt
>> suggested by Arnd.
>> - Add Tested-by from Jhon and pk.
>>
>> Changes since [v6]
>> - Fix build errors reported by kbuild test robot.
>>
>> Changes since [v5]
>> - Move reserve_crashkernel_low() into kernel/crash_core.c.
>> - Dele
Re: [PATCH v13 6/8] arm64: kdump: reimplement crashkernel=X
Hi Baoquan,
On 2020/11/11 9:59, Baoquan He wrote:
> On 10/31/20 at 03:44pm, Chen Zhou wrote:
>> There are following issues in arm64 kdump:
>> 1. We use crashkernel=X to reserve crashkernel below 4G, which
>> will fail when there is no enough low memory.
>> 2. If reserving crashkernel above 4G, in this case, crash dump
>> kernel will boot failure because there is no low memory available
>> for allocation.
>> 3. Since commit 1a8e1cef7603 ("arm64: use both ZONE_DMA and ZONE_DMA32"),
>> if the memory reserved for crash dump kernel falled in ZONE_DMA32,
>> the devices in crash dump kernel need to use ZONE_DMA will alloc
>> fail.
>>
>> To solve these issues, change the behavior of crashkernel=X and
>> introduce crashkernel=X,[high,low]. crashkernel=X tries low allocation
>> in DMA zone or DMA32 zone if CONFIG_ZONE_DMA is disabled, and fall back
>> to high allocation if it fails.
>> We can also use "crashkernel=X,high" to select a region above DMA zone,
>> which also tries to allocate at least 256M in DMA zone automatically
>> (or the DMA32 zone if CONFIG_ZONE_DMA is disabled).
>> "crashkernel=Y,low" can be used to allocate specified size low memory.
>>
>> Another minor change, there may be two regions reserved for crash
>> dump kernel, in order to distinct from the high region and make no
>> effect to the use of existing kexec-tools, rename the low region as
>> "Crash kernel (low)".
>>
>> Signed-off-by: Chen Zhou
>> Tested-by: John Donnelly
>> ---
>> arch/arm64/include/asm/kexec.h | 9 +
>> arch/arm64/kernel/setup.c | 13 +++-
>> arch/arm64/mm/init.c | 60 ++
>> arch/arm64/mm/mmu.c| 4 +++
>> kernel/crash_core.c| 8 +++--
>> 5 files changed, 34 insertions(+), 60 deletions(-)
>>
>> diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h
>> index 402d208265a3..79909ae5e22e 100644
>> --- a/arch/arm64/include/asm/kexec.h
>> +++ b/arch/arm64/include/asm/kexec.h
>> @@ -28,7 +28,12 @@
>> /* 2M alignment for crash kernel regions */
>> #define CRASH_ALIGN SZ_2M
>>
>> +#ifdef CONFIG_ZONE_DMA
>> +#define CRASH_ADDR_LOW_MAX arm64_dma_phys_limit
>> +#else
>> #define CRASH_ADDR_LOW_MAX arm64_dma32_phys_limit
>> +#endif
>> +
>> #define CRASH_ADDR_HIGH_MAX MEMBLOCK_ALLOC_ACCESSIBLE
>>
>> #ifndef __ASSEMBLY__
>> @@ -96,6 +101,10 @@ static inline void crash_prepare_suspend(void) {}
>> static inline void crash_post_resume(void) {}
>> #endif
>>
>> +#ifdef CONFIG_KEXEC_CORE
>> +extern void __init reserve_crashkernel(void);
>> +#endif
>> +
>> #ifdef CONFIG_KEXEC_FILE
>> #define ARCH_HAS_KIMAGE_ARCH
>>
>> diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
>> index 133257ffd859..6aff30de8f47 100644
>> --- a/arch/arm64/kernel/setup.c
>> +++ b/arch/arm64/kernel/setup.c
>> @@ -238,7 +238,18 @@ static void __init request_standard_resources(void)
>> kernel_data.end <= res->end)
>> request_resource(res, &kernel_data);
>> #ifdef CONFIG_KEXEC_CORE
>> -/* Userspace will find "Crash kernel" region in /proc/iomem. */
>> +/*
>> + * Userspace will find "Crash kernel" or "Crash kernel (low)"
>> + * region in /proc/iomem.
>> + * In order to distinct from the high region and make no effect
>> + * to the use of existing kexec-tools, rename the low region as
>> + * "Crash kernel (low)".
>> + */
>> +if (crashk_low_res.end && crashk_low_res.start >= res->start &&
>> +crashk_low_res.end <= res->end) {
>> +crashk_low_res.name = "Crash kernel (low)";
>> +request_resource(res, &crashk_low_res);
>> +}
>> if (crashk_res.end && crashk_res.start >= res->start &&
>> crashk_res.end <= res->end)
>> request_resource(res, &crashk_res);
>> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
>> index a07fd8e1f926..888c4f7eadc3 100644
>> --- a/arch/arm64/mm/init.c
>> +++ b/arch/arm64/mm/init.c
>> @@ -34,6 +34,7 @@
>> #include
>> #include
>> #include
>> +#include
>> #include
>> #include
>> #include
>> @@ -62,66 +63,11 @@ EXPORT_SYMBOL(memstart_addr);
>> phys_addr_t arm64_dma_phys_limit __ro_after_init;
>> phys_addr_t arm64_dma32_phys_limit __ro_after_init;
>>
>> -#ifdef CONFIG_KEXEC_CORE
>> -/*
>> - * reserve_crashkernel() - reserves memory for crash kernel
>> - *
>> - * This function reserves memory area given in "crashkernel=" kernel command
>> - * line parameter. The memory reserved is used by dump capture kernel when
>> - * primary kernel is crashing.
>> - */
>> -static void __init reserve_crashkernel(void)
>> -{
>> -unsigned long long crash_base, crash_size;
>> -int ret;
>> -
>> -ret = parse_crashkernel(boot_command_line, memblock_phys_mem_size(),
>> -&crash_size, &crash_base);
>> -/* no c
Re: [PATCH v13 1/8] x86: kdump: replace the hard-coded alignment with macro CRASH_ALIGN
Hi Baoquan,
On 2020/11/11 9:38, Baoquan He wrote:
> On 10/31/20 at 03:44pm, Chen Zhou wrote:
>> Move CRASH_ALIGN to header asm/kexec.h and replace the hard-coded
>> alignment with macro CRASH_ALIGN in function reserve_crashkernel().
> Seems you tell what you have done in this patch, but don't like adding
> several more words to tell why it's done like that. Please see below
> inline comments.
>
> In other patches, I can also see this similar problem.
Thanks for your review. I will update relevant commit messages.
>
>> Suggested-by: Dave Young
>> Signed-off-by: Chen Zhou
>> Tested-by: John Donnelly
>> ---
>> arch/x86/include/asm/kexec.h | 3 +++
>> arch/x86/kernel/setup.c | 5 +
>> 2 files changed, 4 insertions(+), 4 deletions(-)
>>
>> diff --git a/arch/x86/include/asm/kexec.h b/arch/x86/include/asm/kexec.h
>> index 6802c59e8252..8cf9d3fd31c7 100644
>> --- a/arch/x86/include/asm/kexec.h
>> +++ b/arch/x86/include/asm/kexec.h
>> @@ -18,6 +18,9 @@
>>
>> # define KEXEC_CONTROL_CODE_MAX_SIZE2048
>>
>> +/* 2M alignment for crash kernel regions */
>> +#define CRASH_ALIGN SZ_16M
>> +
>> #ifndef __ASSEMBLY__
>>
>> #include
>> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
>> index 84f581c91db4..bf373422dc8a 100644
>> --- a/arch/x86/kernel/setup.c
>> +++ b/arch/x86/kernel/setup.c
>> @@ -395,9 +395,6 @@ static void __init
>> memblock_x86_reserve_range_setup_data(void)
>>
>> #ifdef CONFIG_KEXEC_CORE
>>
>> -/* 16M alignment for crash kernel regions */
>> -#define CRASH_ALIGN SZ_16M
>> -
>> /*
>> * Keep the crash kernel below this limit.
>> *
>> @@ -515,7 +512,7 @@ static void __init reserve_crashkernel(void)
>> } else {
>> unsigned long long start;
>>
>> -start = memblock_phys_alloc_range(crash_size, SZ_1M, crash_base,
>> +start = memblock_phys_alloc_range(crash_size, CRASH_ALIGN,
>> crash_base,
>>crash_base + crash_size);
> Here, SZ_1M is replaced with CRASH_ALIGN which is 16M. I remember I ever
> commented that this had better be told in patch log.
got it.
Thanks,
Chen Zhou
>
>> if (start != crash_base) {
>> pr_info("crashkernel reservation failed - memory is in
>> use.\n");
>> --
>> 2.20.1
>>
>>
>> ___
>> kexec mailing list
>> kexec@lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/kexec
>>
> .
>
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v13 0/8] support reserving crashkernel above 4G on arm64 kdump
Hi all,
Friendly ping...
On 2020/10/31 15:44, Chen Zhou wrote:
> There are following issues in arm64 kdump:
> 1. We use crashkernel=X to reserve crashkernel below 4G, which
> will fail when there is no enough low memory.
> 2. If reserving crashkernel above 4G, in this case, crash dump
> kernel will boot failure because there is no low memory available
> for allocation.
> 3. Since commit 1a8e1cef7603 ("arm64: use both ZONE_DMA and ZONE_DMA32"),
> if the memory reserved for crash dump kernel falled in ZONE_DMA32,
> the devices in crash dump kernel need to use ZONE_DMA will alloc
> fail.
>
> To solve these issues, change the behavior of crashkernel=X.
> crashkernel=X tries low allocation in DMA zone (or the DMA32 zone if
> CONFIG_ZONE_DMA is disabled), and fall back to high allocation if it fails.
>
> We can also use "crashkernel=X,high" to select a high region above
> DMA zone, which also tries to allocate at least 256M low memory in
> DMA zone automatically (or the DMA32 zone if CONFIG_ZONE_DMA is disabled).
> "crashkernel=Y,low" can be used to allocate specified size low memory.
>
> When reserving crashkernel in high memory, some low memory is reserved
> for crash dump kernel devices. So there may be two regions reserved for
> crash dump kernel.
> In order to distinct from the high region and make no effect to the use
> of existing kexec-tools, rename the low region as "Crash kernel (low)",
> and pass the low region by reusing DT property
> "linux,usable-memory-range". We made the low memory region as the last
> range of "linux,usable-memory-range" to keep compatibility with existing
> user-space and older kdump kernels.
>
> Besides, we need to modify kexec-tools:
> arm64: support more than one crash kernel regions(see [1])
>
> Another update is document about DT property 'linux,usable-memory-range':
> schemas: update 'linux,usable-memory-range' node schema(see [2])
>
> This patchset contains the following eight patches:
> 0001-x86-kdump-replace-the-hard-coded-alignment-with-macr.patch
> 0002-x86-kdump-make-the-lower-bound-of-crash-kernel-reser.patch
> 0003-x86-kdump-use-macro-CRASH_ADDR_LOW_MAX-in-functions-.patch
> 0004-x86-kdump-move-reserve_crashkernel-_low-into-crash_c.patch
> 0005-arm64-kdump-introduce-some-macroes-for-crash-kernel-.patch
> 0006-arm64-kdump-reimplement-crashkernel-X.patch
> 0007-arm64-kdump-add-memory-for-devices-by-DT-property-li.patch
> 0008-kdump-update-Documentation-about-crashkernel.patch
>
> 0001-0003 are some x86 cleanups which prepares for making
> functionsreserve_crashkernel[_low]() generic.
> 0004 makes functions reserve_crashkernel[_low]() generic.
> 0005-0006 reimplements arm64 crashkernel=X.
> 0007 adds memory for devices by DT property linux,usable-memory-range.
> 0008 updates the doc.
>
> Changes since [v12]
> - Rebased on top of 5.10-rc1.
> - Keep CRASH_ALIGN as 16M suggested by Dave.
> - Drop patch "kdump: add threshold for the required memory".
> - Add Tested-by from John.
>
> Changes since [v11]
> - Rebased on top of 5.9-rc4.
> - Make the function reserve_crashkernel() of x86 generic.
> Suggested by Catalin, make the function reserve_crashkernel() of x86 generic
> and arm64 use the generic version to reimplement crashkernel=X.
>
> Changes since [v10]
> - Reimplement crashkernel=X suggested by Catalin, Many thanks to Catalin.
>
> Changes since [v9]
> - Patch 1 add Acked-by from Dave.
> - Update patch 5 according to Dave's comments.
> - Update chosen schema.
>
> Changes since [v8]
> - Reuse DT property "linux,usable-memory-range".
> Suggested by Rob, reuse DT property "linux,usable-memory-range" to pass the
> low
> memory region.
> - Fix kdump broken with ZONE_DMA reintroduced.
> - Update chosen schema.
>
> Changes since [v7]
> - Move x86 CRASH_ALIGN to 2M
> Suggested by Dave and do some test, move x86 CRASH_ALIGN to 2M.
> - Update Documentation/devicetree/bindings/chosen.txt.
> Add corresponding documentation to
> Documentation/devicetree/bindings/chosen.txt
> suggested by Arnd.
> - Add Tested-by from Jhon and pk.
>
> Changes since [v6]
> - Fix build errors reported by kbuild test robot.
>
> Changes since [v5]
> - Move reserve_crashkernel_low() into kernel/crash_core.c.
> - Delete crashkernel=X,high.
> - Modify crashkernel=X,low.
> If crashkernel=X,low is specified simultaneously, reserve spcified size low
> memory for crash kdump kernel devices firstly and then reserve memory above
> 4G.
> In addition, rename crashk_low_res as "Crash kernel (low)" for arm64, and then
> pass to crash dump kernel by DT property "linux,low-memory-range".
> - Update Documentation/admin-guide/kdump/kdump.rst.
>
> Changes since [v4]
> - Reimplement memblock_cap_memory_ranges for multiple ranges by Mike.
>
> Changes since [v3]
> - Add memblock_cap_memory_ranges back for multiple ranges.
> - Fix some compiling warnings.
>
> Changes since [v2]
> - Split patch "arm64: kdump: support reserving crashkernel above 4G" as
> two. Put "move reserve_crashkernel_low() into kexec_core.c" in a s
Re: [PATCH v12 0/9] support reserving crashkernel above 4G on arm64 kdump
Hi Bhupesh, On 2020/10/7 15:07, Bhupesh Sharma wrote: > Hi Catalin, > > On Tue, Oct 6, 2020 at 11:30 PM Catalin Marinas > wrote: >> On Mon, Oct 05, 2020 at 11:12:10PM +0530, Bhupesh Sharma wrote: >>> I think my earlier email with the test results on this series bounced >>> off the mailing list server (for some weird reason), but I still see >>> several issues with this patchset. I will add specific issues in the >>> review comments for each patch again, but overall, with a crashkernel >>> size of say 786M, I see the following issue: >>> >>> # cat /proc/cmdline >>> BOOT_IMAGE=(hd7,gpt2)/vmlinuz-5.9.0-rc7+ root=<..snip..> >>> rd.lvm.lv=<..snip..> crashkernel=786M >>> >>> I see two regions of size 786M and 256M reserved in low and high >>> regions respectively, So we reserve a total of 1042M of memory, which >>> is an incorrect behaviour: >>> >>> # dmesg | grep -i crash >>> [0.00] Reserving 256MB of low memory at 2816MB for crashkernel >>> (System low RAM: 768MB) >>> [0.00] Reserving 786MB of memory at 654158MB for crashkernel >>> (System RAM: 130816MB) >>> [0.00] Kernel command line: >>> BOOT_IMAGE=(hd2,gpt2)/vmlinuz-5.9.0-rc7+ >>> root=/dev/mapper/rhel_ampere--hr330a--03-root ro >>> rd.lvm.lv=rhel_ampere-hr330a-03/root rd.lvm.lv=rhel_ampere-hr330a-03/swap >>> crashkernel=786M cma=1024M >>> >>> # cat /proc/iomem | grep -i crash >>> b000-bfff : Crash kernel (low) >>> bfcbe0-bffcff : Crash kernel >> As Chen said, that's the intended behaviour and how x86 works. The >> requested 768M goes in the high range if there's not enough low memory >> and an additional buffer for swiotlb is allocated, hence the low 256M. > I understand, but why 256M (as low) for arm64? x86_64 setups usually > have more system memory available as compared to several commercially > available arm64 setups. So is the intent, just to keep the behavior > similar between arm64 and x86_64? > > Should we have a CONFIG option / bootarg to help one select the max > 'low_size'? Currently the ' low_size' value is calculated as: > > /* > * two parts from kernel/dma/swiotlb.c: > * -swiotlb size: user-specified with swiotlb= or default. > * > * -swiotlb overflow buffer: now hardcoded to 32k. We round it > * to 8M for other buffers that may need to stay low too. Also > * make sure we allocate enough extra low memory so that we > * don't run out of DMA buffers for 32-bit devices. > */ > low_size = max(swiotlb_size_or_default() + (8UL << 20), 256UL << 20); > > Since many arm64 boards ship with swiotlb=0 (turned off) via kernel > bootargs, the low_size, still ends up being 256M in such cases, > whereas this 256M can be used for some other purposes - so should we > be limiting this to 64M and failing the crash kernel allocation > request (gracefully) otherwise? > >> We could (as an additional patch), subtract the 256M from the high >> allocation so that you'd get a low 256M and a high 512M, not sure it's >> worth it. Note that with a "crashkernel=768M,high" option, you still get >> the additional low 256M, otherwise the crashkernel won't be able to >> boot as there's no memory in ZONE_DMA. In the explicit ",high" request >> case, I'm not sure subtracted the 256M is more intuitive. >> In 5.11, we also hope to fix the ZONE_DMA layout for non-RPi4 platforms >> to cover the entire 32-bit address space (i.e. identical to the current >> ZONE_DMA32). >> >>> IMO, we should test this feature more before including this in 5.11 >> Definitely. That's one of the reasons we haven't queued it yet. So any >> help with testing here is appreciated. > Sure, I am running more checks on this series. I will be soon back > with more updates. Sorry to bother you. I am looking forward to your review comments. Thanks, Chen Zhou > > Regards, > Bhupesh > > . > ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v12 9/9] kdump: update Documentation about crashkernel
Hi Catalin, On 2020/10/6 1:19, Catalin Marinas wrote: > On Mon, Sep 07, 2020 at 09:47:45PM +0800, Chen Zhou wrote: >> diff --git a/Documentation/admin-guide/kdump/kdump.rst >> b/Documentation/admin-guide/kdump/kdump.rst >> index 2da65fef2a1c..549611abc581 100644 >> --- a/Documentation/admin-guide/kdump/kdump.rst >> +++ b/Documentation/admin-guide/kdump/kdump.rst > [...] >> @@ -316,8 +325,18 @@ Boot into System Kernel >> kernel will automatically locate the crash kernel image within the >> first 512MB of RAM if X is not given. >> >> - On arm64, use "crashkernel=Y[@X]". Note that the start address of >> - the kernel, X if explicitly specified, must be aligned to 2MiB >> (0x20). >> + On arm64, use "crashkernel=X" to try low allocation in DMA zone, and >> + fall back to high allocation if it fails. And go for high allocation >> + directly if the required size is too large. >> + We can also use "crashkernel=X,high" to select a high region above >> + DMA zone, which also tries to allocate at least 256M low memory in >> + DMA zone automatically. >> + "crashkernel=Y,low" can be used to allocate specified size low memory >> + in DMA zone. >> + For non-RPi4 platforms, change DMA zone memtioned above to DMA32 zone. > I don't think we should mention non-RPi4 explicitly here. I don't even > understand what the suggestion is since the only way is to disable > ZONE_DMA in the kernel config. I'd just stick to ZONE_DMA description > here. How about like this: If the kernel config ZONE_DMA is disabled, just try low allocation in DMA32 zone and high allocation above DMA32 zone. Thanks, Chen Zhou > >> + Use "crashkernel=Y@X" if you really have to reserve memory from >> + specified start address X. Note that the start address of the kernel, >> + X if explicitly specified, must be aligned to 2MiB (0x20). >> >> Load the Dump-capture Kernel >> >> diff --git a/Documentation/admin-guide/kernel-parameters.txt >> b/Documentation/admin-guide/kernel-parameters.txt >> index a1068742a6df..f7df572d8f64 100644 >> --- a/Documentation/admin-guide/kernel-parameters.txt >> +++ b/Documentation/admin-guide/kernel-parameters.txt >> @@ -727,6 +727,10 @@ >> [KNL, X86-64] Select a region under 4G first, and >> fall back to reserve region above 4G when '@offset' >> hasn't been specified. >> +[KNL, arm64] Try low allocation in DMA zone, fall back >> +to high allocation if it fails when '@offset' hasn't >> been >> +specified. For non-RPi4 platforms, change DMA zone to >> +DMA32 zone. > Same here, unclear what "change DMA zone to DMA32 zone" means. > >> See Documentation/admin-guide/kdump/kdump.rst for >> further details. >> >> crashkernel=range1:size1[,range2:size2,...][@offset] >> @@ -743,6 +747,8 @@ >> Otherwise memory region will be allocated below 4G, if >> available. >> It will be ignored if crashkernel=X is specified. >> +[KNL, arm64] range in high memory. >> +Allow kernel to allocate physical memory region from >> top. >> crashkernel=size[KMG],low >> [KNL, X86-64] range under 4G. When crashkernel=X,high >> is passed, kernel could allocate physical memory region >> @@ -751,13 +757,16 @@ >> requires at least 64M+32K low memory, also enough extra >> low memory is needed to make sure DMA buffers for 32-bit >> devices won't run out. Kernel would try to allocate at >> -at least 256M below 4G automatically. >> +least 256M below 4G automatically. >> This one let user to specify own low range under 4G >> for second kernel instead. >> 0: to disable low allocation. >> It will be ignored when crashkernel=X,high is not used >> or memory reserved is below 4G. >> - >> +[KNL, arm64] range in low memory. >> +This one let user to specify a low range in DMA zone for >> +crash dump kernel. For non-RPi4 platforms, change DMA >> zone >> +to DMA32 zone. > And again here. > ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v12 0/9] support reserving crashkernel above 4G on arm64 kdump
Hi Bhupesh, On 2020/10/6 1:42, Bhupesh Sharma wrote: > Hi Catalin, Chen, > > On Mon, Oct 5, 2020 at 10:39 PM Catalin Marinas > wrote: >> On Sat, Sep 12, 2020 at 06:44:29AM -0500, John Donnelly wrote: >>> On 9/7/20 8:47 AM, Chen Zhou wrote: Chen Zhou (9): x86: kdump: move CRASH_ALIGN to 2M x86: kdump: make the lower bound of crash kernel reservation consistent x86: kdump: use macro CRASH_ADDR_LOW_MAX in functions reserve_crashkernel[_low]() x86: kdump: move reserve_crashkernel[_low]() into crash_core.c arm64: kdump: introduce some macroes for crash kernel reservation arm64: kdump: reimplement crashkernel=X kdump: add threshold for the required memory arm64: kdump: add memory for devices by DT property linux,usable-memory-range kdump: update Documentation about crashkernel >> [...] >>> I did a brief unit-test on 5.9-rc4. >>> >>> Please add: >>> >>> Tested-by: John Donnelly >> Thanks for testing. >> >>> This activity is over a year old. It needs accepted. >> It's getting there, hopefully in 5.11. There are some minor tweaks to >> address. > I think my earlier email with the test results on this series bounced > off the mailing list server (for some weird reason), but I still see > several issues with this patchset. I will add specific issues in the > review comments for each patch again, but overall, with a crashkernel > size of say 786M, I see the following issue: > > # cat /proc/cmdline > BOOT_IMAGE=(hd7,gpt2)/vmlinuz-5.9.0-rc7+ root=<..snip..> > rd.lvm.lv=<..snip..> crashkernel=786M > > I see two regions of size 786M and 256M reserved in low and high > regions respectively, So we reserve a total of 1042M of memory, which > is an incorrect behaviour: > > # dmesg | grep -i crash > [0.00] Reserving 256MB of low memory at 2816MB for crashkernel > (System low RAM: 768MB) > [0.00] Reserving 786MB of memory at 654158MB for crashkernel > (System RAM: 130816MB) > [0.00] Kernel command line: > BOOT_IMAGE=(hd2,gpt2)/vmlinuz-5.9.0-rc7+ > root=/dev/mapper/rhel_ampere--hr330a--03-root ro > rd.lvm.lv=rhel_ampere-hr330a-03/root > rd.lvm.lv=rhel_ampere-hr330a-03/swap crashkernel=786M cma=1024M > > # cat /proc/iomem | grep -i crash > b000-bfff : Crash kernel (low) > bfcbe0-bffcff : Crash kernel > > IMO, we should test this feature more before including this in 5.11 Thanks for you test. This behavior is what we what. What is the correct behavior you think? Besides, this feature is been tested by John and PK, and i test for various parameters. We may miss something, any comments are welcome. Thanks, Chen Zhou > > Thanks, > Bhupesh > > . > ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v12 6/9] arm64: kdump: reimplement crashkernel=X
On 2020/10/6 1:16, Catalin Marinas wrote:
> On Mon, Sep 07, 2020 at 09:47:42PM +0800, Chen Zhou wrote:
>> diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
>> index 53acbeca4f57..1b24072f2bae 100644
>> --- a/arch/arm64/kernel/setup.c
>> +++ b/arch/arm64/kernel/setup.c
>> @@ -238,7 +238,18 @@ static void __init request_standard_resources(void)
>> kernel_data.end <= res->end)
>> request_resource(res, &kernel_data);
>> #ifdef CONFIG_KEXEC_CORE
>> -/* Userspace will find "Crash kernel" region in /proc/iomem. */
>> +/*
>> + * Userspace will find "Crash kernel" or "Crash kernel (low)"
>> + * region in /proc/iomem.
>> + * In order to distinct from the high region and make no effect
>> + * to the use of existing kexec-tools, rename the low region as
>> + * "Crash kernel (low)".
>> + */
>> +if (crashk_low_res.end && crashk_low_res.start >= res->start &&
>> +crashk_low_res.end <= res->end) {
>> +crashk_low_res.name = "Crash kernel (low)";
>> +request_resource(res, &crashk_low_res);
>> +}
> With the changes in this series (including the above), how do the
> current kexec-tools behave? Do they pick just the high region and the
> loaded kernel will subsequently fail to boot?
Yes，just pick the high region and will boot fail if low memory is needed.
Thanks,
Chen Zhou
>
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v12 7/9] kdump: add threshold for the required memory
On 2020/10/6 1:12, Catalin Marinas wrote: > On Mon, Sep 07, 2020 at 09:47:43PM +0800, Chen Zhou wrote: >> diff --git a/kernel/crash_core.c b/kernel/crash_core.c >> index 3f735cb37ace..d11d597a470d 100644 >> --- a/kernel/crash_core.c >> +++ b/kernel/crash_core.c >> @@ -378,6 +378,15 @@ int __init reserve_crashkernel_low(void) >> } >> >> #if defined(CONFIG_X86) || defined(CONFIG_ARM64) >> + >> +/* >> + * Add a threshold for required memory size of crashkernel. If required >> memory >> + * size is greater than threshold, just go for high allocation directly. The >> + * value of threshold is set as half of the total low memory. >> + */ >> +#define REQUIRED_MEMORY_THRESHOLD (memblock_mem_size(CRASH_ADDR_LOW_MAX >> >> \ >> +PAGE_SHIFT) >> 1) >> + >> #ifdef CONFIG_KEXEC_CORE >> /* >> * reserve_crashkernel() - reserves memory for crash kernel >> @@ -422,7 +431,7 @@ void __init reserve_crashkernel(void) >> * So try low memory first and fall back to high memory >> * unless "crashkernel=size[KMG],high" is specified. >> */ >> -if (!high) >> +if (!high && crash_size <= REQUIRED_MEMORY_THRESHOLD) >> crash_base = memblock_find_in_range(CRASH_ALIGN, >> CRASH_ADDR_LOW_MAX, >> crash_size, CRASH_ALIGN); > Since any change now is affecting the x86 semantics slightly, I'd > suggest you drop this patch. We can add it later if needed, once the > core changes are in. Ok, i will drop this patch in next version. Thanks, Chen Zhou > > Thinking about this, if one requires a crashkernel reservation that > allocates all of the ZONE_DMA, it would probably be noticed and explicit > ,high/,low options can be used. > > Note that we are also trying to make ZONE_DMA full 32-bit on non-RPi4 > hardware. > ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v12 3/9] x86: kdump: use macro CRASH_ADDR_LOW_MAX in functions reserve_crashkernel[_low]()
Hi Catalin,
On 2020/9/18 16:59, chenzhou wrote:
> Hi Baoquan,
>
> On 2020/9/18 15:25, Baoquan He wrote:
>> Hi,
>>
>> On 09/07/20 at 09:47pm, Chen Zhou wrote:
>>> To make the functions reserve_crashkernel[_low]() as generic,
>>> replace some hard-coded numbers with macro CRASH_ADDR_LOW_MAX.
>>>
>>> Signed-off-by: Chen Zhou
>>> ---
>>> arch/x86/kernel/setup.c | 11 ++-
>>> 1 file changed, 6 insertions(+), 5 deletions(-)
>>>
>>> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
>>> index d7fd90c52dae..71a6a6e7ca5b 100644
>>> --- a/arch/x86/kernel/setup.c
>>> +++ b/arch/x86/kernel/setup.c
>>> @@ -430,7 +430,7 @@ static int __init reserve_crashkernel_low(void)
>>> unsigned long total_low_mem;
>>> int ret;
>>>
>>> - total_low_mem = memblock_mem_size(1UL << (32 - PAGE_SHIFT));
>>> + total_low_mem = memblock_mem_size(CRASH_ADDR_LOW_MAX >> PAGE_SHIFT);
>> Just note that the replacement has been done in another patch from Mike
>> Rapoport, partially. He seems to have done reserve_crashkernel_low()
>> part, there's one left in reserve_crashkernel(), you might want to check
>> that.
>>
>> Mike's patch which is from a patchset has been merged into Andrew's next
>> tree.
>>
>> commit 6e50f7672ffa362e9bd4bc0c0d2524ed872828c5
>> Author: Mike Rapoport
>> Date: Wed Aug 26 15:22:32 2020 +1000
>>
>> x86/setup: simplify reserve_crashkernel()
As Baoquan said, some functions have been changed in the next tree,
if i need to rebase on top of the next tree.
Thanks,
Chen Zhou
> Yeah, the function reserve_crashkernel() has been changed in the next tree.
> Thanks for your review and reminder.
>
> Thanks,
> Chen Zhou
>>>
>>> /* crashkernel=Y,low */
>>> ret = parse_crashkernel_low(boot_command_line, total_low_mem,
>>> &low_size, &base);
>>> @@ -451,7 +451,7 @@ static int __init reserve_crashkernel_low(void)
>>> return 0;
>>> }
>>>
>>> - low_base = memblock_find_in_range(CRASH_ALIGN, 1ULL << 32, low_size,
>>> CRASH_ALIGN);
>>> + low_base = memblock_find_in_range(CRASH_ALIGN, CRASH_ADDR_LOW_MAX,
>>> low_size, CRASH_ALIGN);
>>> if (!low_base) {
>>> pr_err("Cannot reserve %ldMB crashkernel low memory, please try
>>> smaller size.\n",
>>>(unsigned long)(low_size >> 20));
>>> @@ -504,8 +504,9 @@ static void __init reserve_crashkernel(void)
>>> if (!crash_base) {
>>> /*
>>> * Set CRASH_ADDR_LOW_MAX upper bound for crash memory,
>>> -* crashkernel=x,high reserves memory over 4G, also allocates
>>> -* 256M extra low memory for DMA buffers and swiotlb.
>>> +* crashkernel=x,high reserves memory over CRASH_ADDR_LOW_MAX,
>>> +* also allocates 256M extra low memory for DMA buffers
>>> +* and swiotlb.
>>> * But the extra memory is not required for all machines.
>>> * So try low memory first and fall back to high memory
>>> * unless "crashkernel=size[KMG],high" is specified.
>>> @@ -539,7 +540,7 @@ static void __init reserve_crashkernel(void)
>>> return;
>>> }
>>>
>>> - if (crash_base >= (1ULL << 32) && reserve_crashkernel_low()) {
>>> + if (crash_base >= CRASH_ADDR_LOW_MAX && reserve_crashkernel_low()) {
>>> memblock_free(crash_base, crash_size);
>>> return;
>>> }
>>> --
>>> 2.20.1
>>>
>> .
>>
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v12 3/9] x86: kdump: use macro CRASH_ADDR_LOW_MAX in functions reserve_crashkernel[_low]()
Hi Baoquan,
On 2020/9/18 15:25, Baoquan He wrote:
> Hi,
>
> On 09/07/20 at 09:47pm, Chen Zhou wrote:
>> To make the functions reserve_crashkernel[_low]() as generic,
>> replace some hard-coded numbers with macro CRASH_ADDR_LOW_MAX.
>>
>> Signed-off-by: Chen Zhou
>> ---
>> arch/x86/kernel/setup.c | 11 ++-
>> 1 file changed, 6 insertions(+), 5 deletions(-)
>>
>> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
>> index d7fd90c52dae..71a6a6e7ca5b 100644
>> --- a/arch/x86/kernel/setup.c
>> +++ b/arch/x86/kernel/setup.c
>> @@ -430,7 +430,7 @@ static int __init reserve_crashkernel_low(void)
>> unsigned long total_low_mem;
>> int ret;
>>
>> -total_low_mem = memblock_mem_size(1UL << (32 - PAGE_SHIFT));
>> +total_low_mem = memblock_mem_size(CRASH_ADDR_LOW_MAX >> PAGE_SHIFT);
> Just note that the replacement has been done in another patch from Mike
> Rapoport, partially. He seems to have done reserve_crashkernel_low()
> part, there's one left in reserve_crashkernel(), you might want to check
> that.
>
> Mike's patch which is from a patchset has been merged into Andrew's next
> tree.
>
> commit 6e50f7672ffa362e9bd4bc0c0d2524ed872828c5
> Author: Mike Rapoport
> Date: Wed Aug 26 15:22:32 2020 +1000
>
> x86/setup: simplify reserve_crashkernel()
Yeah, the function reserve_crashkernel() has been changed in the next tree.
Thanks for your review and reminder.
Thanks,
Chen Zhou
>
>>
>> /* crashkernel=Y,low */
>> ret = parse_crashkernel_low(boot_command_line, total_low_mem,
>> &low_size, &base);
>> @@ -451,7 +451,7 @@ static int __init reserve_crashkernel_low(void)
>> return 0;
>> }
>>
>> -low_base = memblock_find_in_range(CRASH_ALIGN, 1ULL << 32, low_size,
>> CRASH_ALIGN);
>> +low_base = memblock_find_in_range(CRASH_ALIGN, CRASH_ADDR_LOW_MAX,
>> low_size, CRASH_ALIGN);
>> if (!low_base) {
>> pr_err("Cannot reserve %ldMB crashkernel low memory, please try
>> smaller size.\n",
>> (unsigned long)(low_size >> 20));
>> @@ -504,8 +504,9 @@ static void __init reserve_crashkernel(void)
>> if (!crash_base) {
>> /*
>> * Set CRASH_ADDR_LOW_MAX upper bound for crash memory,
>> - * crashkernel=x,high reserves memory over 4G, also allocates
>> - * 256M extra low memory for DMA buffers and swiotlb.
>> + * crashkernel=x,high reserves memory over CRASH_ADDR_LOW_MAX,
>> + * also allocates 256M extra low memory for DMA buffers
>> + * and swiotlb.
>> * But the extra memory is not required for all machines.
>> * So try low memory first and fall back to high memory
>> * unless "crashkernel=size[KMG],high" is specified.
>> @@ -539,7 +540,7 @@ static void __init reserve_crashkernel(void)
>> return;
>> }
>>
>> -if (crash_base >= (1ULL << 32) && reserve_crashkernel_low()) {
>> +if (crash_base >= CRASH_ADDR_LOW_MAX && reserve_crashkernel_low()) {
>> memblock_free(crash_base, crash_size);
>> return;
>> }
>> --
>> 2.20.1
>>
> .
>
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v12 3/9] x86: kdump: use macro CRASH_ADDR_LOW_MAX in functions reserve_crashkernel[_low]()
Hi Dave,
On 2020/9/18 11:01, Dave Young wrote:
> On 09/07/20 at 09:47pm, Chen Zhou wrote:
>> To make the functions reserve_crashkernel[_low]() as generic,
>> replace some hard-coded numbers with macro CRASH_ADDR_LOW_MAX.
>>
>> Signed-off-by: Chen Zhou
>> ---
>> arch/x86/kernel/setup.c | 11 ++-
>> 1 file changed, 6 insertions(+), 5 deletions(-)
>>
>> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
>> index d7fd90c52dae..71a6a6e7ca5b 100644
>> --- a/arch/x86/kernel/setup.c
>> +++ b/arch/x86/kernel/setup.c
>> @@ -430,7 +430,7 @@ static int __init reserve_crashkernel_low(void)
>> unsigned long total_low_mem;
>> int ret;
>>
>> -total_low_mem = memblock_mem_size(1UL << (32 - PAGE_SHIFT));
>> +total_low_mem = memblock_mem_size(CRASH_ADDR_LOW_MAX >> PAGE_SHIFT);
> total_low_mem != CRASH_ADDR_LOW_MAX
I just replace the magic number with macro, no other change.
Besides, function memblock_mem_size(limit_pfn) will compute the memory size
according to the actual system ram.
Thanks,
Chen Zhou
>
>>
>> /* crashkernel=Y,low */
>> ret = parse_crashkernel_low(boot_command_line, total_low_mem,
>> &low_size, &base);
> The param total_low_mem is for dynamically change crash_size according
> to system ram size.
>
> Is above change a must for your arm64 patches?
See above.
>
>> @@ -451,7 +451,7 @@ static int __init reserve_crashkernel_low(void)
>> return 0;
>> }
>>
>> -low_base = memblock_find_in_range(CRASH_ALIGN, 1ULL << 32, low_size,
>> CRASH_ALIGN);
>> +low_base = memblock_find_in_range(CRASH_ALIGN, CRASH_ADDR_LOW_MAX,
>> low_size, CRASH_ALIGN);
>> if (!low_base) {
>> pr_err("Cannot reserve %ldMB crashkernel low memory, please try
>> smaller size.\n",
>> (unsigned long)(low_size >> 20));
>> @@ -504,8 +504,9 @@ static void __init reserve_crashkernel(void)
>> if (!crash_base) {
>> /*
>> * Set CRASH_ADDR_LOW_MAX upper bound for crash memory,
>> - * crashkernel=x,high reserves memory over 4G, also allocates
>> - * 256M extra low memory for DMA buffers and swiotlb.
>> + * crashkernel=x,high reserves memory over CRASH_ADDR_LOW_MAX,
>> + * also allocates 256M extra low memory for DMA buffers
>> + * and swiotlb.
>> * But the extra memory is not required for all machines.
>> * So try low memory first and fall back to high memory
>> * unless "crashkernel=size[KMG],high" is specified.
>> @@ -539,7 +540,7 @@ static void __init reserve_crashkernel(void)
>> return;
>> }
>>
>> -if (crash_base >= (1ULL << 32) && reserve_crashkernel_low()) {
>> +if (crash_base >= CRASH_ADDR_LOW_MAX && reserve_crashkernel_low()) {
>> memblock_free(crash_base, crash_size);
>> return;
>> }
>> --
>> 2.20.1
>>
> .
>
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v12 0/9] support reserving crashkernel above 4G on arm64 kdump
On 2020/9/7 21:47, Chen Zhou wrote:
> There are following issues in arm64 kdump:
> 1. We use crashkernel=X to reserve crashkernel below 4G, which
> will fail when there is no enough low memory.
> 2. If reserving crashkernel above 4G, in this case, crash dump
> kernel will boot failure because there is no low memory available
> for allocation.
> 3. Since commit 1a8e1cef7603 ("arm64: use both ZONE_DMA and ZONE_DMA32"),
> if the memory reserved for crash dump kernel falled in ZONE_DMA32,
> the devices in crash dump kernel need to use ZONE_DMA will alloc
> fail.
>
> To solve these issues, change the behavior of crashkernel=X.
> crashkernel=X tries low allocation in DMA zone, and fall back to
> high allocation if it fails.
> If requized size X is too large and leads to very little low memory
> in DMA zone after low allocation, the system may not work normally.
> So add a threshold and go for high allocation directly if the required
> size is too large. The value of threshold is set as the half of
> the low memory.
>
> We can also use "crashkernel=X,high" to select a high region above
> DMA zone, which also tries to allocate at least 256M low memory in
> DMA zone automatically.
> "crashkernel=Y,low" can be used to allocate specified size low memory.
> For non-RPi4 platforms, change DMA zone memtioned above to DMA32 zone.
>
> When reserving crashkernel in high memory, some low memory is reserved
> for crash dump kernel devices. So there may be two regions reserved for
> crash dump kernel.
> In order to distinct from the high region and make no effect to the use
> of existing kexec-tools, rename the low region as "Crash kernel (low)",
> and pass the low region by reusing DT property
> "linux,usable-memory-range". We made the low memory region as the last
> range of "linux,usable-memory-range" to keep compatibility with existing
> user-space and older kdump kernels.
>
> Besides, we need to modify kexec-tools:
> arm64: support more than one crash kernel regions(see [1])
>
> Another update is document about DT property 'linux,usable-memory-range':
> schemas: update 'linux,usable-memory-range' node schema(see [2])
>
> This patchset contains the following nine patches:
> 0001-x86-kdump-move-CRASH_ALIGN-to-2M.patch
> 0002-x86-kdump-make-the-lower-bound-of-crash-kernel-reser.patch
> 0003-x86-kdump-use-macro-CRASH_ADDR_LOW_MAX-in-functions-.patch
> 0004-x86-kdump-move-reserve_crashkernel-_low-into-crash_c.patch
> 0005-arm64-kdump-introduce-some-macroes-for-crash-kernel-.patch
> 0006-arm64-kdump-reimplement-crashkernel-X.patch
> 0007-kdump-add-threshold-for-the-required-memory.patch
> 0008-arm64-kdump-add-memory-for-devices-by-DT-property-li.patch
> 0009-kdump-update-Documentation-about-crashkernel.patch
>
> 0001-0003 are some x86 cleanups which prepares for making
> functionsreserve_crashkernel[_low]() generic.
>
> 0004 makes functions reserve_crashkernel[_low]() generic.
> 0005-0006 reimplements crashkernel=X.
> 0007 adds threshold for the required memory.
> 0008 adds memory for devices by DT property linux,usable-memory-range.
> 0009 updates the doc.
Hi Catalin and Dave,
Any other suggestions about this patchset? Let me know if you have any
questions.
Thanks,
Chen Zhou
>
> Changes since [v11]
> - Rebased on top of 5.9-rc4.
> - Make the function reserve_crashkernel() of x86 generic.
> Suggested by Catalin, make the function reserve_crashkernel() of x86 generic
> and arm64 use the generic version to reimplement crashkernel=X.
>
> Changes since [v10]
> - Reimplement crashkernel=X suggested by Catalin, Many thanks to Catalin.
>
> Changes since [v9]
> - Patch 1 add Acked-by from Dave.
> - Update patch 5 according to Dave's comments.
> - Update chosen schema.
>
> Changes since [v8]
> - Reuse DT property "linux,usable-memory-range".
> Suggested by Rob, reuse DT property "linux,usable-memory-range" to pass the
> low
> memory region.
> - Fix kdump broken with ZONE_DMA reintroduced.
> - Update chosen schema.
>
> Changes since [v7]
> - Move x86 CRASH_ALIGN to 2M
> Suggested by Dave and do some test, move x86 CRASH_ALIGN to 2M.
> - Update Documentation/devicetree/bindings/chosen.txt.
> Add corresponding documentation to
> Documentation/devicetree/bindings/chosen.txt
> suggested by Arnd.
> - Add Tested-by from Jhon and pk.
>
> Changes since [v6]
> - Fix build errors reported by kbuild test robot.
>
> Changes since [v5]
> - Move reserve_crashkernel_low() into kernel/crash_core.c.
> - Delete crashkernel=X,high.
> - Modify crashkernel=X,low.
> If crashkernel=X,low is specified simultaneously, reserve spcified size low
> memory for crash kdump kernel devices firstly and then reserve memory above
> 4G.
> In addition, rename crashk_low_res as "Crash kernel (low)" for arm64, and then
> pass to crash dump kernel by DT property "linux,low-memory-range".
> - Update Documentation/admin-guide/kdump/kdump.rst.
>
> Changes since [v4]
> - Reimplement memblock_cap_memory_ranges for multiple ranges by Mike.
>
> Changes since [v3]
> - Add me
Re: [PATCH v12 1/9] x86: kdump: move CRASH_ALIGN to 2M
On 2020/9/8 9:21, Dave Young wrote:
> Hi,
>
> On 09/07/20 at 09:47pm, Chen Zhou wrote:
>> CONFIG_PHYSICAL_ALIGN can be selected from 2M to 16M and default
>> value is 2M, so move CRASH_ALIGN to 2M, with smaller value reservation
>> can have more chance to succeed.
> Seems still some misunderstanding about the change :( I'm sorry if I
> did not explain it clearly.
>
> Previously I missed the PHYSICAL_ALIGN can change according to .config
> I mean we should change the value to CONFIG_PHYSICAL_ALIGN for X86
> And I suggest to move back to keep using 16M. And do not change it in
> this series.
Hi Dave,
Sorry, i misunderstood about this.
Ok, this patch will keep the value of CRASH_ALIGN as it is,
just move CRASH_ALIGN to header asm/kexec.h and replace the hard-coded alignment
with macro CRASH_ALIGN in function reserve_crashkernel().
Thanks,
Chen Zhou
>
>> And replace the hard-coded alignment with macro CRASH_ALIGN in function
>> reserve_crashkernel().
>>
>> Suggested-by: Dave Young
>> Signed-off-by: Chen Zhou
>> ---
>> arch/x86/include/asm/kexec.h | 3 +++
>> arch/x86/kernel/setup.c | 5 +
>> 2 files changed, 4 insertions(+), 4 deletions(-)
>>
>> diff --git a/arch/x86/include/asm/kexec.h b/arch/x86/include/asm/kexec.h
>> index 6802c59e8252..83f200dd54a1 100644
>> --- a/arch/x86/include/asm/kexec.h
>> +++ b/arch/x86/include/asm/kexec.h
>> @@ -18,6 +18,9 @@
>>
>> # define KEXEC_CONTROL_CODE_MAX_SIZE2048
>>
>> +/* 2M alignment for crash kernel regions */
>> +#define CRASH_ALIGN SZ_2M
>> +
>> #ifndef __ASSEMBLY__
>>
>> #include
>> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
>> index 3511736fbc74..296294ad0dd8 100644
>> --- a/arch/x86/kernel/setup.c
>> +++ b/arch/x86/kernel/setup.c
>> @@ -402,9 +402,6 @@ static void __init
>> memblock_x86_reserve_range_setup_data(void)
>>
>> #ifdef CONFIG_KEXEC_CORE
>>
>> -/* 16M alignment for crash kernel regions */
>> -#define CRASH_ALIGN SZ_16M
>> -
>> /*
>> * Keep the crash kernel below this limit.
>> *
>> @@ -530,7 +527,7 @@ static void __init reserve_crashkernel(void)
>>
>> start = memblock_find_in_range(crash_base,
>> crash_base + crash_size,
>> - crash_size, 1 << 20);
>> + crash_size, CRASH_ALIGN);
>> if (start != crash_base) {
>> pr_info("crashkernel reservation failed - memory is in
>> use.\n");
>> return;
>> --
>> 2.20.1
>>
> Thanks
> Dave
>
>
> .
>
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v11 3/5] arm64: kdump: reimplement crashkernel=X
On 2020/9/4 12:16, Dave Young wrote:
> On 09/04/20 at 12:02pm, chenzhou wrote:
>>
>> On 2020/9/4 11:10, Dave Young wrote:
>>> On 09/04/20 at 11:04am, Dave Young wrote:
>>>> On 09/03/20 at 07:26pm, chenzhou wrote:
>>>>> Hi Catalin,
>>>>>
>>>>>
>>>>> On 2020/9/3 1:09, Catalin Marinas wrote:
>>>>>> On Sat, Aug 01, 2020 at 09:08:54PM +0800, Chen Zhou wrote:
>>>>>>> There are following issues in arm64 kdump:
>>>>>>> 1. We use crashkernel=X to reserve crashkernel below 4G, which
>>>>>>> will fail when there is no enough low memory.
>>>>>>> 2. If reserving crashkernel above 4G, in this case, crash dump
>>>>>>> kernel will boot failure because there is no low memory available
>>>>>>> for allocation.
>>>>>>> 3. Since commit 1a8e1cef7603 ("arm64: use both ZONE_DMA and
>>>>>>> ZONE_DMA32"),
>>>>>>> if the memory reserved for crash dump kernel falled in ZONE_DMA32,
>>>>>>> the devices in crash dump kernel need to use ZONE_DMA will alloc
>>>>>>> fail.
>>>>>>>
>>>>>>> To solve these issues, change the behavior of crashkernel=X.
>>>>>>> crashkernel=X tries low allocation in ZONE_DMA, and fall back to
>>>>>>> high allocation if it fails.
>>>>>>>
>>>>>>> If requized size X is too large and leads to very little free memory
>>>>>>> in ZONE_DMA after low allocation, the system may not work normally.
>>>>>>> So add a threshold and go for high allocation directly if the required
>>>>>>> size is too large. The value of threshold is set as the half of
>>>>>>> the low memory.
>>>>>>>
>>>>>>> If crash_base is outside ZONE_DMA, try to allocate at least 256M in
>>>>>>> ZONE_DMA automatically. "crashkernel=Y,low" can be used to allocate
>>>>>>> specified size low memory.
>>>>>> Except for the threshold to keep zone ZONE_DMA memory,
>>>>>> reserve_crashkernel() looks very close to the x86 version. Shall we try
>>>>>> to make this generic as well? In the first instance, you could avoid the
>>>>>> threshold check if it takes an explicit ",high" option.
>>>>> Ok, i will try to do this.
>>>>>
>>>>> I look into the function reserve_crashkernel() of x86 and found the start
>>>>> address is
>>>>> CRASH_ALIGN in function memblock_find_in_range(), which is different with
>>>>> arm64.
>>>>>
>>>>> I don't figure out why is CRASH_ALIGN in x86, is there any specific
>>>>> reason?
>>>> Hmm, took another look at the option CONFIG_PHYSICAL_ALIGN
>>>> config PHYSICAL_ALIGN
>>>> hex "Alignment value to which kernel should be aligned"
>>>> default "0x20"
>>>> range 0x2000 0x100 if X86_32
>>>> range 0x20 0x100 if X86_64
>>>>
>>>> According to above, I think the 16M should come from the largest value
>>>> But the default value is 2M, with smaller value reservation can have
>>>> more chance to succeed.
>>>>
>>>> It seems we still need arch specific CRASH_ALIGN, but the initial
>>>> version you added the #ifdef for different arches, can you move the
>>>> macro to arch specific headers?
>>> And just keep the x86 align value as is, I can try to change the x86
>>> value later to CONFIG_PHYSICAL_ALIGN, in this way this series can be
>>> cleaner.
>> Ok. I have no question about the value of macro CRASH_ALIGN,
>> instead the lower bound of memblock_find_in_range().
>>
>> For x86, in reserve_crashkernel()，restrict the lower bound of the range to
>> CRASH_ALIGN,
>> ...
>> crash_base = memblock_find_in_range(CRASH_ALIGN,
>> CRASH_ADDR_LOW_MAX,
>> crash_size, CRASH_ALIGN);
>> ...
>>
>> in reserve_crashkernel_low()，with no this restriction.
>> ...
>> low_base = memblock_find_in_range(0, 1ULL << 32, low_size, CRASH_ALIGN);
>> ...
>>
>> How about all making memblock_find_in_range() search from the start of
>> memory?
>> If it is ok, i will do like this in the generic version.
> I feel starting with CRASH_ALIGN sounds better, can you just search from
> CRASH_ALIGN in generic version?
ok.
>
> Thanks
> Dave
>
>
> .
>
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v11 3/5] arm64: kdump: reimplement crashkernel=X
On 2020/9/4 11:10, Dave Young wrote:
> On 09/04/20 at 11:04am, Dave Young wrote:
>> On 09/03/20 at 07:26pm, chenzhou wrote:
>>> Hi Catalin,
>>>
>>>
>>> On 2020/9/3 1:09, Catalin Marinas wrote:
>>>> On Sat, Aug 01, 2020 at 09:08:54PM +0800, Chen Zhou wrote:
>>>>> There are following issues in arm64 kdump:
>>>>> 1. We use crashkernel=X to reserve crashkernel below 4G, which
>>>>> will fail when there is no enough low memory.
>>>>> 2. If reserving crashkernel above 4G, in this case, crash dump
>>>>> kernel will boot failure because there is no low memory available
>>>>> for allocation.
>>>>> 3. Since commit 1a8e1cef7603 ("arm64: use both ZONE_DMA and ZONE_DMA32"),
>>>>> if the memory reserved for crash dump kernel falled in ZONE_DMA32,
>>>>> the devices in crash dump kernel need to use ZONE_DMA will alloc
>>>>> fail.
>>>>>
>>>>> To solve these issues, change the behavior of crashkernel=X.
>>>>> crashkernel=X tries low allocation in ZONE_DMA, and fall back to
>>>>> high allocation if it fails.
>>>>>
>>>>> If requized size X is too large and leads to very little free memory
>>>>> in ZONE_DMA after low allocation, the system may not work normally.
>>>>> So add a threshold and go for high allocation directly if the required
>>>>> size is too large. The value of threshold is set as the half of
>>>>> the low memory.
>>>>>
>>>>> If crash_base is outside ZONE_DMA, try to allocate at least 256M in
>>>>> ZONE_DMA automatically. "crashkernel=Y,low" can be used to allocate
>>>>> specified size low memory.
>>>> Except for the threshold to keep zone ZONE_DMA memory,
>>>> reserve_crashkernel() looks very close to the x86 version. Shall we try
>>>> to make this generic as well? In the first instance, you could avoid the
>>>> threshold check if it takes an explicit ",high" option.
>>> Ok, i will try to do this.
>>>
>>> I look into the function reserve_crashkernel() of x86 and found the start
>>> address is
>>> CRASH_ALIGN in function memblock_find_in_range(), which is different with
>>> arm64.
>>>
>>> I don't figure out why is CRASH_ALIGN in x86, is there any specific reason?
>> Hmm, took another look at the option CONFIG_PHYSICAL_ALIGN
>> config PHYSICAL_ALIGN
>> hex "Alignment value to which kernel should be aligned"
>> default "0x20"
>> range 0x2000 0x100 if X86_32
>> range 0x20 0x100 if X86_64
>>
>> According to above, I think the 16M should come from the largest value
>> But the default value is 2M, with smaller value reservation can have
>> more chance to succeed.
>>
>> It seems we still need arch specific CRASH_ALIGN, but the initial
>> version you added the #ifdef for different arches, can you move the
>> macro to arch specific headers?
> And just keep the x86 align value as is, I can try to change the x86
> value later to CONFIG_PHYSICAL_ALIGN, in this way this series can be
> cleaner.
Ok. I have no question about the value of macro CRASH_ALIGN,
instead the lower bound of memblock_find_in_range().
For x86, in reserve_crashkernel()，restrict the lower bound of the range to
CRASH_ALIGN,
...
crash_base = memblock_find_in_range(CRASH_ALIGN,
CRASH_ADDR_LOW_MAX,
crash_size, CRASH_ALIGN);
...
in reserve_crashkernel_low()，with no this restriction.
...
low_base = memblock_find_in_range(0, 1ULL << 32, low_size, CRASH_ALIGN);
...
How about all making memblock_find_in_range() search from the start of memory?
If it is ok, i will do like this in the generic version.
Thanks,
Chen Zhou
>
>> Thanks
>> Dave
>
> .
>
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v11 3/5] arm64: kdump: reimplement crashkernel=X
On 2020/9/3 19:26, chenzhou wrote:
> Hi Catalin,
>
>
> On 2020/9/3 1:09, Catalin Marinas wrote:
>> On Sat, Aug 01, 2020 at 09:08:54PM +0800, Chen Zhou wrote:
>>> There are following issues in arm64 kdump:
>>> 1. We use crashkernel=X to reserve crashkernel below 4G, which
>>> will fail when there is no enough low memory.
>>> 2. If reserving crashkernel above 4G, in this case, crash dump
>>> kernel will boot failure because there is no low memory available
>>> for allocation.
>>> 3. Since commit 1a8e1cef7603 ("arm64: use both ZONE_DMA and ZONE_DMA32"),
>>> if the memory reserved for crash dump kernel falled in ZONE_DMA32,
>>> the devices in crash dump kernel need to use ZONE_DMA will alloc
>>> fail.
>>>
>>> To solve these issues, change the behavior of crashkernel=X.
>>> crashkernel=X tries low allocation in ZONE_DMA, and fall back to
>>> high allocation if it fails.
>>>
>>> If requized size X is too large and leads to very little free memory
>>> in ZONE_DMA after low allocation, the system may not work normally.
>>> So add a threshold and go for high allocation directly if the required
>>> size is too large. The value of threshold is set as the half of
>>> the low memory.
>>>
>>> If crash_base is outside ZONE_DMA, try to allocate at least 256M in
>>> ZONE_DMA automatically. "crashkernel=Y,low" can be used to allocate
>>> specified size low memory.
>> Except for the threshold to keep zone ZONE_DMA memory,
>> reserve_crashkernel() looks very close to the x86 version. Shall we try
>> to make this generic as well? In the first instance, you could avoid the
>> threshold check if it takes an explicit ",high" option.
> Ok, i will try to do this.
>
> I look into the function reserve_crashkernel() of x86 and found the start
> address is
> CRASH_ALIGN in function memblock_find_in_range(), which is different with
> arm64.
>
> I don't figure out why is CRASH_ALIGN in x86, is there any specific reason?
Besides, in function reserve_crashkernel_low() of x86, the start address is 0.
>
> Thanks,
> Chen Zhou
>
>
>
> ___
> linux-arm-kernel mailing list
> linux-arm-ker...@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>
> .
>
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v11 5/5] kdump: update Documentation about crashkernel
On 2020/9/3 1:13, Catalin Marinas wrote: > On Sat, Aug 01, 2020 at 09:08:56PM +0800, Chen Zhou wrote: >> diff --git a/Documentation/admin-guide/kdump/kdump.rst >> b/Documentation/admin-guide/kdump/kdump.rst >> index 2da65fef2a1c..4b58f97351d5 100644 >> --- a/Documentation/admin-guide/kdump/kdump.rst >> +++ b/Documentation/admin-guide/kdump/kdump.rst >> @@ -299,7 +299,15 @@ Boot into System Kernel >> "crashkernel=64M@16M" tells the system kernel to reserve 64 MB of memory >> starting at physical address 0x0100 (16MB) for the dump-capture >> kernel. >> >> - On x86 and x86_64, use "crashkernel=64M@16M". >> + On x86 use "crashkernel=64M@16M". >> + >> + On x86_64, use "crashkernel=X" to select a region under 4G first, and >> + fall back to reserve region above 4G. >> + We can also use "crashkernel=X,high" to select a region above 4G, which >> + also tries to allocate at least 256M below 4G automatically and >> + "crashkernel=Y,low" can be used to allocate specified size low memory. >> + Use "crashkernel=Y@X" if you really have to reserve memory from specified >> + start address X. >> >> On ppc64, use "crashkernel=128M@32M". >> >> @@ -316,8 +324,15 @@ Boot into System Kernel >> kernel will automatically locate the crash kernel image within the >> first 512MB of RAM if X is not given. >> >> - On arm64, use "crashkernel=Y[@X]". Note that the start address of >> - the kernel, X if explicitly specified, must be aligned to 2MiB >> (0x20). >> + On arm64, use "crashkernel=X" to try low allocation in ZONE_DMA, and >> + fall back to high allocation if it fails. And go for high allocation >> + directly if the required size is too large. If crash_base is outside > I wouldn't mention crash_base in the admin guide. That's an > implementation detail really and admins are not supposed to read the > source code to make sense of the documentation. ZONE_DMA is also a > kernel internal, so you'd need to define what it is for arm64. At least > the DMA and DMA32 zones are printed during kernel boot. Ok, i will fix this in next version. > >> + ZONE_DMA, try to allocate at least 256M in ZONE_DMA automatically. >> + "crashkernel=Y,low" can be used to allocate specified size low memory. >> + For non-RPi4 platforms, change ZONE_DMA memtioned above to ZONE_DMA32. >> + Use "crashkernel=Y@X" if you really have to reserve memory from >> + specified start address X. Note that the start address of the kernel, >> + X if explicitly specified, must be aligned to 2MiB (0x20). ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v11 3/5] arm64: kdump: reimplement crashkernel=X
Hi Catalin,
On 2020/9/3 1:09, Catalin Marinas wrote:
> On Sat, Aug 01, 2020 at 09:08:54PM +0800, Chen Zhou wrote:
>> There are following issues in arm64 kdump:
>> 1. We use crashkernel=X to reserve crashkernel below 4G, which
>> will fail when there is no enough low memory.
>> 2. If reserving crashkernel above 4G, in this case, crash dump
>> kernel will boot failure because there is no low memory available
>> for allocation.
>> 3. Since commit 1a8e1cef7603 ("arm64: use both ZONE_DMA and ZONE_DMA32"),
>> if the memory reserved for crash dump kernel falled in ZONE_DMA32,
>> the devices in crash dump kernel need to use ZONE_DMA will alloc
>> fail.
>>
>> To solve these issues, change the behavior of crashkernel=X.
>> crashkernel=X tries low allocation in ZONE_DMA, and fall back to
>> high allocation if it fails.
>>
>> If requized size X is too large and leads to very little free memory
>> in ZONE_DMA after low allocation, the system may not work normally.
>> So add a threshold and go for high allocation directly if the required
>> size is too large. The value of threshold is set as the half of
>> the low memory.
>>
>> If crash_base is outside ZONE_DMA, try to allocate at least 256M in
>> ZONE_DMA automatically. "crashkernel=Y,low" can be used to allocate
>> specified size low memory.
> Except for the threshold to keep zone ZONE_DMA memory,
> reserve_crashkernel() looks very close to the x86 version. Shall we try
> to make this generic as well? In the first instance, you could avoid the
> threshold check if it takes an explicit ",high" option.
Ok, i will try to do this.
I look into the function reserve_crashkernel() of x86 and found the start
address is
CRASH_ALIGN in function memblock_find_in_range(), which is different with arm64.
I don't figure out why is CRASH_ALIGN in x86, is there any specific reason?
Thanks,
Chen Zhou
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v11 5/5] kdump: update Documentation about crashkernel
Hi Catalin,
On 2020/8/19 20:03, Dave Young wrote:
> On 08/18/20 at 03:07pm, chenzhou wrote:
>>
>> On 2020/8/10 14:03, Dave Young wrote:
>>> Hi,
>>>
>>>>> Previously I remember we talked about to use similar logic as X86, but I
>>>>> remember you mentioned on some arm64 platform there could be no low
>>>>> memory at all. Is this not a problem now for the fallback? Just be
>>>>> curious, thanks for the update, for the common part looks good.
>>>> Hi Dave,
>>>>
>>>> Did you mean this discuss: https://lkml.org/lkml/2019/12/27/122?
>>> I meant about this reply instead :)
>>> https://lkml.org/lkml/2020/1/16/616
>> Hi Dave,
>>
>> Sorry for not repley in time, I was on holiday last week.
> Hi, no problem, thanks for following up.
>
>> The platform James mentioned may exist for which have no devices and need no
>> low memory.
>> For our arm64 server platform, there are some devices and need low memory.
>>
>> I got it. For the platform with no low memory, reserving crashkernel will
>> always fail.
>> How about like this:
> I think the question should leave to Catalin or James, I have no
> suggestion about this:)
Any suggestions about this?
Thanks,
Chen Zhou
>
>> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
>> index a8e34d97a894..4df18c7ea438 100644
>> --- a/arch/arm64/mm/init.c
>> +++ b/arch/arm64/mm/init.c
>> @@ -147,7 +147,7 @@ static void __init reserve_crashkernel(void)
>> }
>> memblock_reserve(crash_base, crash_size);
>>
>> - if (crash_base >= CRASH_ADDR_LOW_MAX) {
>> + if (memstart_addr < CRASH_ADDR_LOW_MAX && crash_base >=
>> CRASH_ADDR_LOW_MAX) {
>> const char *rename = "Crash kernel (low)";
>>
>> if (reserve_crashkernel_low()) {
>>
>>
>> Thanks,
>> Chen Zhou
>>
>>> Thanks
>>> Dave
>>>
>>>
>>> .
>>>
>>
>
> .
>
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v11 5/5] kdump: update Documentation about crashkernel
On 2020/8/10 14:03, Dave Young wrote:
> Hi,
>
>>> Previously I remember we talked about to use similar logic as X86, but I
>>> remember you mentioned on some arm64 platform there could be no low
>>> memory at all. Is this not a problem now for the fallback? Just be
>>> curious, thanks for the update, for the common part looks good.
>> Hi Dave,
>>
>> Did you mean this discuss: https://lkml.org/lkml/2019/12/27/122?
> I meant about this reply instead :)
> https://lkml.org/lkml/2020/1/16/616
Hi Dave,
Sorry for not repley in time, I was on holiday last week.
The platform James mentioned may exist for which have no devices and need no
low memory.
For our arm64 server platform, there are some devices and need low memory.
I got it. For the platform with no low memory, reserving crashkernel will
always fail.
How about like this:
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index a8e34d97a894..4df18c7ea438 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -147,7 +147,7 @@ static void __init reserve_crashkernel(void)
}
memblock_reserve(crash_base, crash_size);
- if (crash_base >= CRASH_ADDR_LOW_MAX) {
+ if (memstart_addr < CRASH_ADDR_LOW_MAX && crash_base >=
CRASH_ADDR_LOW_MAX) {
const char *rename = "Crash kernel (low)";
if (reserve_crashkernel_low()) {
Thanks,
Chen Zhou
>
> Thanks
> Dave
>
>
> .
>
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v11 5/5] kdump: update Documentation about crashkernel
On 2020/8/8 18:02, Dave Young wrote: > On 08/01/20 at 09:08pm, Chen Zhou wrote: >> Now the behavior of crashkernel=X has been changed, which tries low >> allocation in ZONE_DMA, and fall back to high allocation if it fails. >> >> If requized size X is too large and leads to very little free memory >> in ZONE_DMA after low allocation, the system may not work well. >> So add a threshold and go for high allocation directly if the required >> size is too large. The threshold is set as the half of low memory. >> >> If crash_base is outside ZONE_DMA, try to allocate at least 256M in >> ZONE_DMA automatically. "crashkernel=Y,low" can be used to allocate >> specified size low memory. For non-RPi4 platforms, change ZONE_DMA >> memtioned above to ZONE_DMA32. >> >> So update the Documentation. >> >> Signed-off-by: Chen Zhou >> --- >> Documentation/admin-guide/kdump/kdump.rst | 21 --- >> .../admin-guide/kernel-parameters.txt | 11 -- >> 2 files changed, 27 insertions(+), 5 deletions(-) >> >> diff --git a/Documentation/admin-guide/kdump/kdump.rst >> b/Documentation/admin-guide/kdump/kdump.rst >> index 2da65fef2a1c..4b58f97351d5 100644 >> --- a/Documentation/admin-guide/kdump/kdump.rst >> +++ b/Documentation/admin-guide/kdump/kdump.rst >> @@ -299,7 +299,15 @@ Boot into System Kernel >> "crashkernel=64M@16M" tells the system kernel to reserve 64 MB of memory >> starting at physical address 0x0100 (16MB) for the dump-capture >> kernel. >> >> - On x86 and x86_64, use "crashkernel=64M@16M". >> + On x86 use "crashkernel=64M@16M". >> + >> + On x86_64, use "crashkernel=X" to select a region under 4G first, and >> + fall back to reserve region above 4G. >> + We can also use "crashkernel=X,high" to select a region above 4G, which >> + also tries to allocate at least 256M below 4G automatically and >> + "crashkernel=Y,low" can be used to allocate specified size low memory. >> + Use "crashkernel=Y@X" if you really have to reserve memory from specified >> + start address X. >> >> On ppc64, use "crashkernel=128M@32M". >> >> @@ -316,8 +324,15 @@ Boot into System Kernel >> kernel will automatically locate the crash kernel image within the >> first 512MB of RAM if X is not given. >> >> - On arm64, use "crashkernel=Y[@X]". Note that the start address of >> - the kernel, X if explicitly specified, must be aligned to 2MiB >> (0x20). >> + On arm64, use "crashkernel=X" to try low allocation in ZONE_DMA, and >> + fall back to high allocation if it fails. And go for high allocation >> + directly if the required size is too large. If crash_base is outside >> + ZONE_DMA, try to allocate at least 256M in ZONE_DMA automatically. >> + "crashkernel=Y,low" can be used to allocate specified size low memory. >> + For non-RPi4 platforms, change ZONE_DMA memtioned above to ZONE_DMA32. >> + Use "crashkernel=Y@X" if you really have to reserve memory from >> + specified start address X. Note that the start address of the kernel, >> + X if explicitly specified, must be aligned to 2MiB (0x20). >> >> Load the Dump-capture Kernel >> >> diff --git a/Documentation/admin-guide/kernel-parameters.txt >> b/Documentation/admin-guide/kernel-parameters.txt >> index fb95fad81c79..d1b6016850d6 100644 >> --- a/Documentation/admin-guide/kernel-parameters.txt >> +++ b/Documentation/admin-guide/kernel-parameters.txt >> @@ -722,6 +722,10 @@ >> [KNL, x86_64] select a region under 4G first, and >> fall back to reserve region above 4G when '@offset' >> hasn't been specified. >> +[KNL, arm64] Try low allocation in ZONE_DMA, fall back >> +to high allocation if it fails when '@offset' hasn't >> been >> +specified. For non-RPi4 platforms, change ZONE_DMA to >> +ZONE_DMA32. >> See Documentation/admin-guide/kdump/kdump.rst for >> further details. >> >> crashkernel=range1:size1[,range2:size2,...][@offset] >> @@ -746,13 +750,16 @@ >> requires at least 64M+32K low memory, also enough extra >> low memory is needed to make sure DMA buffers for 32-bit >> devices won't run out. Kernel would try to allocate at >> -at least 256M below 4G automatically. >> +least 256M below 4G automatically. >> This one let user to specify own low range under 4G >> for second kernel instead. >> 0: to disable low allocation. >> It will be ignored when crashkernel=X,high is not used >> or memory reserved is below 4G. >> - >> +[KNL, arm64] range under 4G. >> +This one let user to specify a low range in ZONE_DMA for >> +crash du
Re: [PATCH v10 4/5] arm64: kdump: fix kdump broken with ZONE_DMA reintroduced
Hi Catalin, On 2020/7/29 23:20, Catalin Marinas wrote: > On Wed, Jul 29, 2020 at 10:14:32PM +0800, chenzhou wrote: >> On 2020/7/29 19:58, Catalin Marinas wrote: >>> On Wed, Jul 29, 2020 at 11:52:39AM +0800, chenzhou wrote: >>>> How about like this: >>>> 1. For ZONE_DMA issue, use Bhupesh's solution, keep the crashkernel= >>>>behaviour to ZONE_DMA allocations. >>>> 2. For this patch series, make the reserve_crashkernel_low() to >>>>ZONE_DMA allocations. >>> So you mean rebasing your series on top of Bhupesh's? I guess you can >>> combine the two, I really don't care which way as long as we fix both >>> issues and agree on the crashkernel= semantics. I think with some tweaks >>> we can go with your series alone. >>> >>> IIUC from the x86 code (especially the part you #ifdef'ed out for >>> arm64), if ",low" is not passed (so just standard crashkernel=X), it >>> still allocates sufficient low memory for the swiotlb in ZONE_DMA. The >>> rest can go in a high region. Why can't we do something similar on >>> arm64? Of course, you can keep the ",low" argument for explicit >>> allocation but I don't want to mandate it. >> It is a good idea to combine the two. >> >> For parameter crashkernel=X, we do like this: >> 1. allocate some low memory in ZONE_DMA(or ZONE_DMA32 if CONFIG_ZONE_DMA=n) >> 2. allocate X size memory in a high region >> >> ",low" argument can be used to specify the low memory. >> >> Do i understand correctly? > Yes, although we could follow the x86 approach: > > 1. Try low (ZONE_DMA for arm64) allocation, fallback to high allocation >if it fails. > > 2. If crash_base is outside ZONE_DMA, call reserve_crashkernel_low() >which either honours the ,low option or allocates some small amount >in ZONE_DMA. > > If at some point we have platforms failing step 2, we'll look at > changing ZONE_DMA to the full 4GB on non-RPi4 platforms. > > It looks to me like x86 ignores the ,low option if the first step > managed to get some low memory. Shall we do the same on arm64? Yes, we could do like this. > >>> So with an implicit ZONE_DMA allocation similar to the x86 one, we >>> probably don't need Bhupesh's series at all. In addition, we can limit >>> crashkernel= to the first 4G with a fall-back to high like x86 (not sure >>> if memblock_find_in_range() is guaranteed to search in ascending order). >>> I don't think we need an explicit ",high" annotation. >>> >>> So with the above, just a crashkernel=1G gives you at least 256MB in >>> ZONE_DMA followed by the rest anywhere, with a preference for >>> ZONE_DMA32. This way we can also keep the reserve_crashkernel_low() >>> mostly intact from x86 (less #ifdef's). >> Yes. We can let crashkernel=X try to reserve low memory and fall back to >> use high memory >> if failing to find a low range. > The only question is whether we need to preserve some more ZONE_DMA on > the current system. If for example we pass a crashkernel=512M and some > cma=, we may end up with very little free memory in ZONE_DMA. That's > mostly an issue for RPi4 since other platforms would work with > ZONE_DMA32. We could add a threshold and go for high allocation directly > if the required size is too large. Ok. I will think about the threshold in the next version and make the value be 1/2 or 1/3 of the ZONE_DMA. > >> About the function reserve_crashkernel_low(), if we put it in arch/arm64, >> there is some common >> code with x86_64. Some suggestions about this? > If we can use this function almost intact, just move it in a common > place. But if it gets sprinkled with #ifdef CONFIG_ARM64, I'd rather > duplicate it. I'd still prefer to move it to a common place if possible. > > You can go a step further and also move the x86 reserve_crashkernel() to > common code. I don't think there a significant difference between arm64 > and x86 here. You'd have to define arch-specific specific > CRASH_ADDR_LOW_MAX etc. I will take these into account and send the next version recently. > > Also patches moving code should not have any functional change. The > CRASH_ALIGN change from 16M to 2M on x86 should be a separate patch as > it needs to be acked by the x86 maintainers (IIRC, Ingo only acked the > function move if there was no functional change; CRASH_ALIGN is used for > the start address, not just alignment, on x86). > Thanks, Chen Zhou ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v10 4/5] arm64: kdump: fix kdump broken with ZONE_DMA reintroduced
Hi Catalin, On 2020/7/29 19:58, Catalin Marinas wrote: > Hi Chen, > > On Wed, Jul 29, 2020 at 11:52:39AM +0800, chenzhou wrote: >> On 2020/7/28 1:30, Catalin Marinas wrote: >>> Anyway, there are two series solving slightly different issues with >>> kdump reservations: >>> >>> 1. This series which relaxes the crashkernel= allocation to go anywhere >>>in the accessible space while having a dedicated crashkernel=X,low >>>option for ZONE_DMA. >>> >>> 2. Bhupesh's series [1] forcing crashkernel=X allocations only from >>>ZONE_DMA. >>> >>> For RPi4 support, we limited ZONE_DMA allocations to the 1st GB. >>> Existing crashkernel= uses may no longer work, depending on where the >>> allocation falls. Option (2) above is a quick fix assuming that the >>> crashkernel reservation is small enough. What's a typical crashkernel >>> option here? That series is probably more prone to reservation failures. >>> >>> Option (1), i.e. this series, doesn't solve the problem raised by >>> Bhupesh unless one uses the crashkernel=X,low argument. It can actually >>> make it worse even for ZONE_DMA32 since the allocation can go above 4G >>> (assuming that we change the ZONE_DMA configuration to only limit it to >>> 1GB on RPi4). >>> >>> I'm more inclined to keep the crashkernel= behaviour to ZONE_DMA >>> allocations. If this is too small for typical kdump, we can look into >>> expanding ZONE_DMA to 4G on non-RPi4 hardware (we had patches on the >>> list). In addition, if Chen thinks allocations above 4G are still needed >>> or if RPi4 needs a sufficiently large crashkernel=, I'd rather have a >>> ",high" option to explicitly require such access. >> Thanks for your reply and exhaustive explanation. >> >> In our ARM servers, we need to to reserve a large chunk for kdump(512M >> or 1G), there is no enough low memory. So we proposed this patch >> series "support reserving crashkernel above 4G on arm64 kdump" In >> April 2019. > Trying to go through the discussions last year, hopefully things get > clearer. > > So prior to the ZONE_DMA change, you still couldn't reserve 1G in the > first 4GB? It shouldn't be sparsely populated during early boot. Yes, we prior to the ZONE_DMA change, you still couldn't reserve 1G/512M in the first 4GB. The memory reported by the bios may be splitted by some "reserved" entries. Like this: ... 2f126000-2fbf : reserved 2fc0-396a : System RAM 30de8000-30de9fff : reserved 30dec000-30decfff : reserved 30df2000-30df2fff : reserved 30e2-30e4 : reserved 3962-3968 : reserved 396b-3974 : reserved 3975-397a : System RAM 397b-398f : reserved 3990-3990 : System RAM 3990-3990 : reserved ... > >> I introduce parameters "crashkernel=X,[high,low]" as x86_64 does in earlier >> versions. >> Suggested by James, to simplify, we call reserve_crashkernel_low() at the >> beginning of >> reserve_crashkernel() and then relax the arm64_dma32_phys_limit if >> reserve_crashkernel_low() >> allocated something. >> That is, just the parameter "crashkernel=X,low" is ok and i deleted >> "crashkernel=X,high". > The problem I see is that with your patches we diverge from x86 > behaviour (and the arm64 behaviour prior to the ZONE_DMA reduction) as > we now require that crashkernel=X,low is always passed if you want > something in ZONE_DMA (and you do want, otherwise the crashdump kernel > fails to boot). > > My main requirement is that crashkernel=X, without any suffix, still > works which I don't think is guaranteed with your patches (well, > ignoring RPi4 ZONE_DMA). Bhupesh's series is a quick fix but doesn't > solve your large allocation requirements (that may have worked prior to > the ZONE_DMA change). The main purpose of this series is to solve the large allocation requirements. Before the DMA_ZONE, both the original crashkernel=X and large allocation with my patches work well. With the DMA_ZONE, both the original crashkernel=X and large allocation with my patches may fail to boot. Both need to think about the DMA_ZONE. > >> After the ZONE_DMA introduced in December 2019, the issue occurred as >> you said above. In fact, we didn't have RPi4 machine. > You don't even need to have a RPi4 machine, ZONE_DMA has been set to 1GB > unconditionally. And while we could move it back to 4GB on non-RPi4 > hardware, I'd rather have a solution that fixes kdump for RPi4 as well. > >> Orig
Re: [PATCH v10 4/5] arm64: kdump: fix kdump broken with ZONE_DMA reintroduced
Hi Catalin,
On 2020/7/28 1:30, Catalin Marinas wrote:
> On Fri, Jul 03, 2020 at 11:58:15AM +0800, Chen Zhou wrote:
>> commit 1a8e1cef7603 ("arm64: use both ZONE_DMA and ZONE_DMA32")
>> broken the arm64 kdump. If the memory reserved for crash dump kernel
>> falled in ZONE_DMA32, the devices in crash dump kernel need to use
>> ZONE_DMA will alloc fail.
>>
>> This patch addressed the above issue based on "reserving crashkernel
>> above 4G". Originally, we reserve low memory below 4G, and now just need
>> to adjust memory limit to arm64_dma_phys_limit in reserve_crashkernel_low
>> if ZONE_DMA is enabled. That is, if there are devices need to use ZONE_DMA
>> in crash dump kernel, it is a good choice to use parameters
>> "crashkernel=X crashkernel=Y,low".
>>
>> Signed-off-by: Chen Zhou
>> ---
>> kernel/crash_core.c | 7 ++-
>> 1 file changed, 6 insertions(+), 1 deletion(-)
>>
>> diff --git a/kernel/crash_core.c b/kernel/crash_core.c
>> index a7580d291c37..e8ecbbc761a3 100644
>> --- a/kernel/crash_core.c
>> +++ b/kernel/crash_core.c
>> @@ -320,6 +320,7 @@ int __init reserve_crashkernel_low(void)
>> unsigned long long base, low_base = 0, low_size = 0;
>> unsigned long total_low_mem;
>> int ret;
>> +phys_addr_t crash_max = 1ULL << 32;
>>
>> total_low_mem = memblock_mem_size(1UL << (32 - PAGE_SHIFT));
>>
>> @@ -352,7 +353,11 @@ int __init reserve_crashkernel_low(void)
>> return 0;
>> }
>>
>> -low_base = memblock_find_in_range(0, 1ULL << 32, low_size, CRASH_ALIGN);
>> +#ifdef CONFIG_ARM64
>> +if (IS_ENABLED(CONFIG_ZONE_DMA))
>> +crash_max = arm64_dma_phys_limit;
>> +#endif
>> +low_base = memblock_find_in_range(0, crash_max, low_size, CRASH_ALIGN);
>> if (!low_base) {
>> pr_err("Cannot reserve %ldMB crashkernel low memory, please try
>> smaller size.\n",
>> (unsigned long)(low_size >> 20));
> Given the number of #ifdefs we end up with in this function, I think
> it's better to simply copy to the code to arch/arm64 and tailor it
> accordingly.
>
> Anyway, there are two series solving slightly different issues with
> kdump reservations:
>
> 1. This series which relaxes the crashkernel= allocation to go anywhere
>in the accessible space while having a dedicated crashkernel=X,low
>option for ZONE_DMA.
>
> 2. Bhupesh's series [1] forcing crashkernel=X allocations only from
>ZONE_DMA.
>
> For RPi4 support, we limited ZONE_DMA allocations to the 1st GB.
> Existing crashkernel= uses may no longer work, depending on where the
> allocation falls. Option (2) above is a quick fix assuming that the
> crashkernel reservation is small enough. What's a typical crashkernel
> option here? That series is probably more prone to reservation failures.
>
> Option (1), i.e. this series, doesn't solve the problem raised by
> Bhupesh unless one uses the crashkernel=X,low argument. It can actually
> make it worse even for ZONE_DMA32 since the allocation can go above 4G
> (assuming that we change the ZONE_DMA configuration to only limit it to
> 1GB on RPi4).
>
> I'm more inclined to keep the crashkernel= behaviour to ZONE_DMA
> allocations. If this is too small for typical kdump, we can look into
> expanding ZONE_DMA to 4G on non-RPi4 hardware (we had patches on the
> list). In addition, if Chen thinks allocations above 4G are still needed
> or if RPi4 needs a sufficiently large crashkernel=, I'd rather have a
> ",high" option to explicitly require such access.
Thanks for your reply and exhaustive explanation.
In our ARM servers, we need to to reserve a large chunk for kdump(512M or 1G),
there is no enough low memory. So we proposed this patch series
"support reserving crashkernel above 4G on arm64 kdump" In April 2019.
I introduce parameters "crashkernel=X,[high,low]" as x86_64 does in earlier
versions.
Suggested by James, to simplify, we call reserve_crashkernel_low() at the
beginning of
reserve_crashkernel() and then relax the arm64_dma32_phys_limit if
reserve_crashkernel_low()
allocated something.
That is, just the parameter "crashkernel=X,low" is ok and i deleted
"crashkernel=X,high".
After the ZONE_DMA introduced in December 2019, the issue occurred as you said
above.
In fact, we didn't have RPi4 machine. Originally, i suggested to fix this based
on this patch series
and used the dedicated option.
According to your clarify, for typical kdump, there are other solutions. In
this case,
"keep the crashkernel= behaviour to ZONE_DMA allocations" looks much better.
How about like this:
1. For ZONE_DMA issue, use Bhupesh's solution, keep the crashkernel= behaviour
to ZONE_DMA allocations.
2. For this patch series, make the reserve_crashkernel_low() to ZONE_DMA
allocations.
Thanks,
Chen Zhou
> [1] http://lists.infradead.org/pipermail/kexec/2020-July/020777.html
>
___
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/list
Re: [PATCH v10 0/5] support reserving crashkernel above 4G on arm64 kdump
Hi Bhupesh,
On 2020/7/3 15:26, Bhupesh Sharma wrote:
> Hi Chen,
>
> On Fri, Jul 3, 2020 at 9:24 AM Chen Zhou wrote:
>> This patch series enable reserving crashkernel above 4G in arm64.
>>
>> There are following issues in arm64 kdump:
>> 1. We use crashkernel=X to reserve crashkernel below 4G, which will fail
>> when there is no enough low memory.
>> 2. Currently, crashkernel=Y@X can be used to reserve crashkernel above 4G,
>> in this case, if swiotlb or DMA buffers are required, crash dump kernel
>> will boot failure because there is no low memory available for allocation.
>> 3. commit 1a8e1cef7603 ("arm64: use both ZONE_DMA and ZONE_DMA32") broken
>> the arm64 kdump. If the memory reserved for crash dump kernel falled in
>> ZONE_DMA32, the devices in crash dump kernel need to use ZONE_DMA will alloc
>> fail.
>>
>> To solve these issues, introduce crashkernel=X,low to reserve specified
>> size low memory.
>> Crashkernel=X tries to reserve memory for the crash dump kernel under
>> 4G. If crashkernel=Y,low is specified simultaneously, reserve spcified
>> size low memory for crash kdump kernel devices firstly and then reserve
>> memory above 4G.
>>
>> When crashkernel is reserved above 4G in memory and crashkernel=X,low
>> is specified simultaneously, kernel should reserve specified size low memory
>> for crash dump kernel devices. So there may be two crash kernel regions, one
>> is below 4G, the other is above 4G.
>> In order to distinct from the high region and make no effect to the use of
>> kexec-tools, rename the low region as "Crash kernel (low)", and pass the
>> low region by reusing DT property "linux,usable-memory-range". We made the
>> low
>> memory region as the last range of "linux,usable-memory-range" to keep
>> compatibility with existing user-space and older kdump kernels.
>>
>> Besides, we need to modify kexec-tools:
>> arm64: support more than one crash kernel regions(see [1])
>>
>> Another update is document about DT property 'linux,usable-memory-range':
>> schemas: update 'linux,usable-memory-range' node schema(see [2])
>>
>> The previous changes and discussions can be retrieved from:
>>
>> Changes since [v9]
>> - Patch 1 add Acked-by from Dave.
>> - Update patch 5 according to Dave's comments.
>> - Update chosen schema.
>>
>> Changes since [v8]
>> - Reuse DT property "linux,usable-memory-range".
>> Suggested by Rob, reuse DT property "linux,usable-memory-range" to pass the
>> low
>> memory region.
>> - Fix kdump broken with ZONE_DMA reintroduced.
>> - Update chosen schema.
>>
>> Changes since [v7]
>> - Move x86 CRASH_ALIGN to 2M
>> Suggested by Dave and do some test, move x86 CRASH_ALIGN to 2M.
>> - Update Documentation/devicetree/bindings/chosen.txt.
>> Add corresponding documentation to
>> Documentation/devicetree/bindings/chosen.txt
>> suggested by Arnd.
>> - Add Tested-by from Jhon and pk.
>>
>> Changes since [v6]
>> - Fix build errors reported by kbuild test robot.
>>
>> Changes since [v5]
>> - Move reserve_crashkernel_low() into kernel/crash_core.c.
>> - Delete crashkernel=X,high.
>> - Modify crashkernel=X,low.
>> If crashkernel=X,low is specified simultaneously, reserve spcified size low
>> memory for crash kdump kernel devices firstly and then reserve memory above
>> 4G.
>> In addition, rename crashk_low_res as "Crash kernel (low)" for arm64, and
>> then
>> pass to crash dump kernel by DT property "linux,low-memory-range".
>> - Update Documentation/admin-guide/kdump/kdump.rst.
>>
>> Changes since [v4]
>> - Reimplement memblock_cap_memory_ranges for multiple ranges by Mike.
>>
>> Changes since [v3]
>> - Add memblock_cap_memory_ranges back for multiple ranges.
>> - Fix some compiling warnings.
>>
>> Changes since [v2]
>> - Split patch "arm64: kdump: support reserving crashkernel above 4G" as
>> two. Put "move reserve_crashkernel_low() into kexec_core.c" in a separate
>> patch.
>>
>> Changes since [v1]:
>> - Move common reserve_crashkernel_low() code into kernel/kexec_core.c.
>> - Remove memblock_cap_memory_ranges() i added in v1 and implement that
>> in fdt_enforce_memory_region().
>> There are at most two crash kernel regions, for two crash kernel regions
>> case, we cap the memory range [min(regs[*].start), max(regs[*].end)]
>> and then remove the memory range in the middle.
>>
>> [1]: http://lists.infradead.org/pipermail/kexec/2020-June/020737.html
>> [2]: https://github.com/robherring/dt-schema/pull/19
>> [v1]: https://lkml.org/lkml/2019/4/2/1174
>> [v2]: https://lkml.org/lkml/2019/4/9/86
>> [v3]: https://lkml.org/lkml/2019/4/9/306
>> [v4]: https://lkml.org/lkml/2019/4/15/273
>> [v5]: https://lkml.org/lkml/2019/5/6/1360
>> [v6]: https://lkml.org/lkml/2019/8/30/142
>> [v7]: https://lkml.org/lkml/2019/12/23/411
>> [v8]: https://lkml.org/lkml/2020/5/21/213
>> [v9]: https://lkml.org/lkml/2020/6/28/73
>>
>> Chen Zhou (5):
>> x86: kdump: move reserve_crashkernel_low() into crash_core.c
>> arm64: kdump: reserve crashkenel above 4G for crash dump kernel
>> arm64: kdump: add
Re: [PATCH 2/2] arm64: Allocate crashkernel always in ZONE_DMA
Hi Bhupesh,
On 2020/7/3 3:22, Bhupesh Sharma wrote:
> Hi Will,
>
> On Thu, Jul 2, 2020 at 1:20 PM Will Deacon wrote:
>> On Thu, Jul 02, 2020 at 03:44:20AM +0530, Bhupesh Sharma wrote:
>>> commit bff3b04460a8 ("arm64: mm: reserve CMA and crashkernel in
>>> ZONE_DMA32") allocates crashkernel for arm64 in the ZONE_DMA32.
>>>
>>> However as reported by Prabhakar, this breaks kdump kernel booting in
>>> ThunderX2 like arm64 systems. I have noticed this on another ampere
>>> arm64 machine. The OOM log in the kdump kernel looks like this:
>>>
>>> [0.240552] DMA: preallocated 128 KiB GFP_KERNEL pool for atomic
>>> allocations
>>> [0.247713] swapper/0: page allocation failure: order:1,
>>> mode:0xcc1(GFP_KERNEL|GFP_DMA), nodemask=(null),cpuset=/,mems_allowed=0
>>> <..snip..>
>>> [0.274706] Call trace:
>>> [0.277170] dump_backtrace+0x0/0x208
>>> [0.280863] show_stack+0x1c/0x28
>>> [0.284207] dump_stack+0xc4/0x10c
>>> [0.287638] warn_alloc+0x104/0x170
>>> [0.291156] __alloc_pages_slowpath.constprop.106+0xb08/0xb48
>>> [0.296958] __alloc_pages_nodemask+0x2ac/0x2f8
>>> [0.301530] alloc_page_interleave+0x20/0x90
>>> [0.305839] alloc_pages_current+0xdc/0xf8
>>> [0.309972] atomic_pool_expand+0x60/0x210
>>> [0.314108] __dma_atomic_pool_init+0x50/0xa4
>>> [0.318504] dma_atomic_pool_init+0xac/0x158
>>> [0.322813] do_one_initcall+0x50/0x218
>>> [0.326684] kernel_init_freeable+0x22c/0x2d0
>>> [0.331083] kernel_init+0x18/0x110
>>> [0.334600] ret_from_fork+0x10/0x18
>>>
>>> This patch limits the crashkernel allocation to the first 1GB of
>>> the RAM accessible (ZONE_DMA), as otherwise we might run into OOM
>>> issues when crashkernel is executed, as it might have been originally
>>> allocated from either a ZONE_DMA32 memory or mixture of memory chunks
>>> belonging to both ZONE_DMA and ZONE_DMA32.
>> How does this interact with this ongoing series:
>>
>> https://lore.kernel.org/r/20200628083458.40066-1-chenzho...@huawei.com
>>
>> (patch 4, in particular)
> Many thanks for having a look at this patchset. I was not aware that
> Chen had sent out a new version.
> I had noted in the v9 review of the high/low range allocation
> that I was working
> on a generic solution (irrespective of the crashkernel, low and high
> range allocation) which resulted in this patchset.
>
> The issue is two-fold: OOPs in memcfg layer (PATCH 1/2, which has been
> Acked-by memcfg maintainer) and OOM in the kdump kernel due to
> crashkernel allocation in ZONE_DMA32 regions(s) which is addressed by
> this PATCH.
>
> I will have a closer look at the v10 patchset Chen shared, but seems
> it needs some rework as per Dave's review comments which he shared
> today.
> IMO, in the meanwhile this patchset can be used to fix the existing
> kdump issue with upstream kernel.
Thanks for your work.
There is no progress on the issue for long time, so i sent my solution in v8
comments
and sent v9 recently.
I think direct limiting the crashkernel in ZONE_DMA isn't a good idea:
1. For parameter "crashkernel=Y", reserving crashkernel in first 1G memory will
increase
the probability of memory allocation failure.
Previous discuss from https://lkml.org/lkml/2019/10/21/725:
"With ZONE_DMA=y, this config will fail to reserve 512M CMA on a server"
2. For parameter "crashkernel=Y@X", limiting the crashkernel in ZONE_DMA is
unreasonable
for someone really want to reserve crashkernel from specified start address.
I have sent v10: https://www.spinics.net/lists/arm-kernel/msg819408.html, any
commets are welcome.
Thanks,
Chen Zhou
>
>>> Fixes: bff3b04460a8 ("arm64: mm: reserve CMA and crashkernel in ZONE_DMA32")
>>> Cc: Johannes Weiner
>>> Cc: Michal Hocko
>>> Cc: Vladimir Davydov
>>> Cc: James Morse
>>> Cc: Mark Rutland
>>> Cc: Will Deacon
>>> Cc: Catalin Marinas
>>> Cc: cgro...@vger.kernel.org
>>> Cc: linux...@kvack.org
>>> Cc: linux-arm-ker...@lists.infradead.org
>>> Cc: linux-ker...@vger.kernel.org
>>> Cc: kexec@lists.infradead.org
>>> Reported-by: Prabhakar Kushwaha
>>> Signed-off-by: Bhupesh Sharma
>>> ---
>>> arch/arm64/mm/init.c | 16 ++--
>>> 1 file changed, 14 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
>>> index 1e93cfc7c47a..02ae4d623802 100644
>>> --- a/arch/arm64/mm/init.c
>>> +++ b/arch/arm64/mm/init.c
>>> @@ -91,8 +91,15 @@ static void __init reserve_crashkernel(void)
>>> crash_size = PAGE_ALIGN(crash_size);
>>>
>>> if (crash_base == 0) {
>>> - /* Current arm64 boot protocol requires 2MB alignment */
>>> - crash_base = memblock_find_in_range(0, arm64_dma32_phys_limit,
>>> + /* Current arm64 boot protocol requires 2MB alignment.
>>> + * Also limit the crashkernel allocation to the first
>>> + * 1GB of the RAM accessible (ZONE_DMA), as
Re: [PATCH v9 5/5] kdump: update Documentation about crashkernel on arm64
Hi Dave, On 2020/7/2 10:59, Dave Young wrote: > Hi Chen, > On 06/28/20 at 04:34pm, Chen Zhou wrote: >> Now we support crashkernel=X,[low] on arm64, update the Documentation. >> We could use parameters "crashkernel=X crashkernel=Y,low" to reserve >> memory above 4G. >> >> Signed-off-by: Chen Zhou >> Tested-by: John Donnelly >> Tested-by: Prabhakar Kushwaha >> --- >> Documentation/admin-guide/kdump/kdump.rst | 13 +++-- >> Documentation/admin-guide/kernel-parameters.txt | 17 +++-- >> 2 files changed, 26 insertions(+), 4 deletions(-) >> >> diff --git a/Documentation/admin-guide/kdump/kdump.rst >> b/Documentation/admin-guide/kdump/kdump.rst >> index 2da65fef2a1c..6ba294d425c9 100644 >> --- a/Documentation/admin-guide/kdump/kdump.rst >> +++ b/Documentation/admin-guide/kdump/kdump.rst >> @@ -299,7 +299,13 @@ Boot into System Kernel >> "crashkernel=64M@16M" tells the system kernel to reserve 64 MB of memory >> starting at physical address 0x0100 (16MB) for the dump-capture >> kernel. >> >> - On x86 and x86_64, use "crashkernel=64M@16M". >> + On x86 use "crashkernel=64M@16M". >> + >> + On x86_64, use "crashkernel=Y[@X]" to select a region under 4G first, and >> + fall back to reserve region above 4G when '@offset' hasn't been >> specified. > Actually crashkernel=Y without the offset works well, I do not see why > we need the offset, it should be some legacy thing. So it should be > better just use the Y without offset here, and just leave a note > somewhere people can use [@X] offset when they really have to. > >> + We can also use "crashkernel=X,high" to select a region above 4G, which >> + also tries to allocate at least 256M below 4G automatically and >> + "crashkernel=Y,low" can be used to allocate specified size low memory. >> >> On ppc64, use "crashkernel=128M@32M". >> >> @@ -316,8 +322,11 @@ Boot into System Kernel >> kernel will automatically locate the crash kernel image within the >> first 512MB of RAM if X is not given. >> >> - On arm64, use "crashkernel=Y[@X]". Note that the start address of >> + On arm64, use "crashkernel=Y[@X]". Note that the start address of >> the kernel, X if explicitly specified, must be aligned to 2MiB >> (0x20). >> + If crashkernel=Z,low is specified simultaneously, reserve spcified size >> + low memory for crash kdump kernel devices firstly and then reserve memory > "devices" seems not very accurate, maybe just drop the "for crash kdump > kernel devices" since it is clear based on the context. > >> + above 4G. >> >> Load the Dump-capture Kernel >> >> diff --git a/Documentation/admin-guide/kernel-parameters.txt >> b/Documentation/admin-guide/kernel-parameters.txt >> index fb95fad81c79..335431a351c0 100644 >> --- a/Documentation/admin-guide/kernel-parameters.txt >> +++ b/Documentation/admin-guide/kernel-parameters.txt >> @@ -722,6 +722,9 @@ >> [KNL, x86_64] select a region under 4G first, and >> fall back to reserve region above 4G when '@offset' >> hasn't been specified. >> +[KNL, arm64] If crashkernel=X,low is specified, reserve >> +spcified size low memory for crash kdump kernel devices > Ditto. > >> +firstly, and then reserve memory above 4G. >> See Documentation/admin-guide/kdump/kdump.rst for >> further details. >> >> crashkernel=range1:size1[,range2:size2,...][@offset] >> @@ -746,13 +749,23 @@ >> requires at least 64M+32K low memory, also enough extra >> low memory is needed to make sure DMA buffers for 32-bit >> devices won't run out. Kernel would try to allocate at >> -at least 256M below 4G automatically. >> +least 256M below 4G automatically. >> This one let user to specify own low range under 4G >> for second kernel instead. >> 0: to disable low allocation. >> It will be ignored when crashkernel=X,high is not used >> or memory reserved is below 4G. >> - >> +[KNL, arm64] range under 4G. >> +This one let user to specify own low range under 4G >> +for crash dump kernel instead. >> +Different with x86_64, kernel allocates specified size > sounds better: > s/Different with/Be different from > > s/allocates/reserves > >> +physical memory region only when this parameter is >> specified >> +instead of trying to allocate at least 256M below 4G > s/allocate/reserve > >> +automatically. >> +This parameter is used along with crashkernel=X when we > Could change the passive sentence to below: > "Use this parameter along with" Thanks f
Re: [PATCH v8 5/5] dt-bindings: chosen: Document linux,low-memory-range for arm64 kdump
Hi James, Rob, On 2020/5/30 0:11, James Morse wrote: > Hi guys, > > On 26/05/2020 22:18, Rob Herring wrote: >> On Fri, May 22, 2020 at 11:24:11AM +0800, chenzhou wrote: >>> On 2020/5/21 21:29, Rob Herring wrote: >>>> On Thu, May 21, 2020 at 3:35 AM Chen Zhou wrote: >>>>> Add documentation for DT property used by arm64 kdump: >>>>> linux,low-memory-range. >>>>> "linux,low-memory-range" is an another memory region used for crash >>>>> dump kernel devices. >>>>> diff --git a/Documentation/devicetree/bindings/chosen.txt >>>>> b/Documentation/devicetree/bindings/chosen.txt >>>>> index 45e79172a646..bfe6fb6976e6 100644 >>>>> --- a/Documentation/devicetree/bindings/chosen.txt >>>>> +++ b/Documentation/devicetree/bindings/chosen.txt >>>>> +linux,low-memory-range >>>>> +-- >>>>> +This property (arm64 only) holds a base address and size, describing a >>>>> +limited region below 4G. Similar to "linux,usable-memory-range", it is >>>>> +an another memory range which may be considered available for use by the >>>>> +kernel. >>>> Why can't you just add a range to "linux,usable-memory-range"? It >>>> shouldn't be hard to figure out which part is below 4G. >>> The comments from James: >>> Won't this break if your kdump kernel doesn't know what the extra >>> parameters are? >>> Or if it expects two ranges, but only gets one? These DT properties should >>> be treated as >>> ABI between kernel versions, we can't really change it like this. >>> >>> I think the 'low' region is an optional-extra, that is never mapped by the >>> first kernel. I >>> think the simplest thing to do is to add an 'linux,low-memory-range' that we >>> memblock_add() after memblock_cap_memory_range() has been called. >>> If its missing, or the new kernel doesn't know what its for, everything >>> keeps working. >> >> I don't think there's a compatibility issue here though. The current >> kernel doesn't care if the property is longer than 1 base+size. It only >> checks if the size is less than 1 base+size. > Aha! I missed that. > > >> And yes, we can rely on >> that implementation detail. It's only an ABI if an existing user >> notices. >> >> Now, if the low memory is listed first, then an older kdump kernel >> would get a different memory range. If that's a problem, then define >> that low memory goes last. > This first entry would need to be the 'crashkernel' range where the kdump > kernel is > placed, otherwise an older kernel won't boot. The rest can be optional > extras, as long as > we are tolerant of it being missing... How about like this: 1. The low memory region remained as "Crash kernel (low)". 2. Userspace will find "Crash kernel" and "Crash kernel (low)" region in /proc/iomem, and add "Crash kernel (low)" as the last range of property "linux,usable-memory-range". Thanks, Chen Zhou > > I'll try and look at the rest of this series on Monday, > > > Thanks, > > James > > . > ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v8 0/5] support reserving crashkernel above 4G on arm64 kdump
On 2020/6/19 10:32, John Donnelly wrote: > > On 6/4/20 12:01 PM, Nicolas Saenz Julienne wrote: >> On Thu, 2020-06-04 at 01:17 +0530, Bhupesh Sharma wrote: >>> Hi All, >>> >>> On Wed, Jun 3, 2020 at 9:03 PM John Donnelly >>> wrote: >>>> >>>>> On Jun 3, 2020, at 8:20 AM, chenzhou wrote: >>>>> >>>>> Hi, >>>>> >>>>> >>>>> On 2020/6/3 19:47, Prabhakar Kushwaha wrote: >>>>>> Hi Chen, >>>>>> >>>>>> On Tue, Jun 2, 2020 at 8:12 PM John Donnelly >>>>>> wrote: >>>>>>> >>>>>>>> On Jun 2, 2020, at 12:38 AM, Prabhakar Kushwaha < >>>>>>>> prabhakar.p...@gmail.com> wrote: >>>>>>>> >>>>>>>> On Tue, Jun 2, 2020 at 3:29 AM John Donnelly < >>>>>>>> john.p.donne...@oracle.com> wrote: >>>>>>>>> Hi . See below ! >>>>>>>>> >>>>>>>>>> On Jun 1, 2020, at 4:02 PM, Bhupesh Sharma >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Hi John, >>>>>>>>>> >>>>>>>>>> On Tue, Jun 2, 2020 at 1:01 AM John Donnelly < >>>>>>>>>> john.p.donne...@oracle.com> wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 6/1/20 7:02 AM, Prabhakar Kushwaha wrote: >>>>>>>>>>>> Hi Chen, >>>>>>>>>>>> >>>>>>>>>>>> On Thu, May 21, 2020 at 3:05 PM Chen Zhou < >>>>>>>>>>>> chenzho...@huawei.com> wrote: >>>>>>>>>>>>> This patch series enable reserving crashkernel above 4G in >>>>>>>>>>>>> arm64. >>>>>>>>>>>>> >>>>>>>>>>>>> There are following issues in arm64 kdump: >>>>>>>>>>>>> 1. We use crashkernel=X to reserve crashkernel below 4G, >>>>>>>>>>>>> which will fail >>>>>>>>>>>>> when there is no enough low memory. >>>>>>>>>>>>> 2. Currently, crashkernel=Y@X can be used to reserve >>>>>>>>>>>>> crashkernel above 4G, >>>>>>>>>>>>> in this case, if swiotlb or DMA buffers are required, >>>>>>>>>>>>> crash dump kernel >>>>>>>>>>>>> will boot failure because there is no low memory available >>>>>>>>>>>>> for allocation. >>>>>>>>>>>>> >>>>>>>>>>>> We are getting "warn_alloc" [1] warning during boot of kdump >>>>>>>>>>>> kernel >>>>>>>>>>>> with bootargs as [2] of primary kernel. >>>>>>>>>>>> This error observed on ThunderX2 ARM64 platform. >>>>>>>>>>>> >>>>>>>>>>>> It is observed with latest upstream tag (v5.7-rc3) with this >>>>>>>>>>>> patch set >>>>>>>>>>>> and >>>>>>>>>>>> >> https://urldefense.com/v3/__https://lists.infradead.org/pipermail/kexec/2020-May/025128.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbiIAAlzu$ >>>>>>>>>>>> Also **without** this patch-set >>>>>>>>>>>> " >>>>>>>>>>>> >> https://urldefense.com/v3/__https://www.spinics.net/lists/arm-kernel/msg806882.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbjC6ujMA$ >>>>>>>>>>>> " >>>>>>>>>>>> >>>>>>>>>>>> This issue comes whenever crashkernel memory is reserved >>>>>>>>>>>> after 0xc000_. >>>>>>>>>>>> More details discussed earlier in >>>>>>>>>>>> >> https://urldefense.com/v3/__https://www.spinics.net
Re: [PATCH v3] arm64: kdump: add another DT property to crash dump kernel's dtb
On 2020/6/6 21:19, Simon Horman wrote: > On Thu, May 21, 2020 at 04:32:57PM +0800, Chen Zhou wrote: >> Currently, there is only one crash kernel region on arm64, we add >> another region "crash kernel low" used for crash dump kernel devices. >> >> To do this, we add DT property "linux,low-memory-range" to crash >> dump kernel's dtb to pass the low region. >> >> Signed-off-by: Chen Zhou >> --- >> For "support reserving crashkernel above 4G on arm64 kdump", we need >> to modify the kexec-tools. >> >> Changes since [v2]: >> - Rebase to latest kexec-tools code. >> >> Changes since [v1]: >> - Add another DT property "linux,low-memory-range" to crash dump kernel's >> dtb to pass the low region instead of reusing "linux,usable-memory-range". >> >> [1]: http://lists.infradead.org/pipermail/kexec/2019-April/022792.html >> [2]: http://lists.infradead.org/pipermail/kexec/2019-August/023569.html > Thanks. > > I think it would be appropriate for me to apply this change to > kexec-tools once the corresponding changes have been accepted to > the kernel. > > Please repost when that is the case. ok. Thanks, Chen Zhou > > . > ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v8 0/5] support reserving crashkernel above 4G on arm64 kdump
Hi, On 2020/6/3 19:47, Prabhakar Kushwaha wrote: > Hi Chen, > > On Tue, Jun 2, 2020 at 8:12 PM John Donnelly > wrote: >> >> >>> On Jun 2, 2020, at 12:38 AM, Prabhakar Kushwaha >>> wrote: >>> >>> On Tue, Jun 2, 2020 at 3:29 AM John Donnelly >>> wrote: Hi . See below ! > On Jun 1, 2020, at 4:02 PM, Bhupesh Sharma wrote: > > Hi John, > > On Tue, Jun 2, 2020 at 1:01 AM John Donnelly > wrote: >> Hi, >> >> >> On 6/1/20 7:02 AM, Prabhakar Kushwaha wrote: >>> Hi Chen, >>> >>> On Thu, May 21, 2020 at 3:05 PM Chen Zhou wrote: This patch series enable reserving crashkernel above 4G in arm64. There are following issues in arm64 kdump: 1. We use crashkernel=X to reserve crashkernel below 4G, which will fail when there is no enough low memory. 2. Currently, crashkernel=Y@X can be used to reserve crashkernel above 4G, in this case, if swiotlb or DMA buffers are required, crash dump kernel will boot failure because there is no low memory available for allocation. >>> We are getting "warn_alloc" [1] warning during boot of kdump kernel >>> with bootargs as [2] of primary kernel. >>> This error observed on ThunderX2 ARM64 platform. >>> >>> It is observed with latest upstream tag (v5.7-rc3) with this patch set >>> and >>> https://urldefense.com/v3/__https://lists.infradead.org/pipermail/kexec/2020-May/025128.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbiIAAlzu$ >>> Also **without** this patch-set >>> "https://urldefense.com/v3/__https://www.spinics.net/lists/arm-kernel/msg806882.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbjC6ujMA$"; >>> >>> This issue comes whenever crashkernel memory is reserved after >>> 0xc000_. >>> More details discussed earlier in >>> https://urldefense.com/v3/__https://www.spinics.net/lists/arm-kernel/msg806882.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbjC6ujMA$ >>> without any >>> solution >>> >>> This patch-set is expected to solve similar kind of issue. >>> i.e. low memory is only targeted for DMA, swiotlb; So above mentioned >>> observation should be considered/fixed. . >>> >>> --pk >>> >>> [1] >>> [ 30.366695] DMI: Cavium Inc. Saber/Saber, BIOS >>> TX2-FW-Release-3.1-build_01-2803-g74253a541a mm/dd/ >>> [ 30.367696] NET: Registered protocol family 16 >>> [ 30.369973] swapper/0: page allocation failure: order:6, >>> mode:0x1(GFP_DMA), nodemask=(null),cpuset=/,mems_allowed=0 >>> [ 30.369980] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.7.0-rc3+ #121 >>> [ 30.369981] Hardware name: Cavium Inc. Saber/Saber, BIOS >>> TX2-FW-Release-3.1-build_01-2803-g74253a541a mm/dd/ >>> [ 30.369984] Call trace: >>> [ 30.369989] dump_backtrace+0x0/0x1f8 >>> [ 30.369991] show_stack+0x20/0x30 >>> [ 30.369997] dump_stack+0xc0/0x10c >>> [ 30.370001] warn_alloc+0x10c/0x178 >>> [ 30.370004] __alloc_pages_slowpath.constprop.111+0xb10/0xb50 >>> [ 30.370006] __alloc_pages_nodemask+0x2b4/0x300 >>> [ 30.370008] alloc_page_interleave+0x24/0x98 >>> [ 30.370011] alloc_pages_current+0xe4/0x108 >>> [ 30.370017] dma_atomic_pool_init+0x44/0x1a4 >>> [ 30.370020] do_one_initcall+0x54/0x228 >>> [ 30.370027] kernel_init_freeable+0x228/0x2cc >>> [ 30.370031] kernel_init+0x1c/0x110 >>> [ 30.370034] ret_from_fork+0x10/0x18 >>> [ 30.370036] Mem-Info: >>> [ 30.370064] active_anon:0 inactive_anon:0 isolated_anon:0 >>> [ 30.370064] active_file:0 inactive_file:0 isolated_file:0 >>> [ 30.370064] unevictable:0 dirty:0 writeback:0 unstable:0 >>> [ 30.370064] slab_reclaimable:34 slab_unreclaimable:4438 >>> [ 30.370064] mapped:0 shmem:0 pagetables:14 bounce:0 >>> [ 30.370064] free:1537719 free_pcp:219 free_cma:0 >>> [ 30.370070] Node 0 active_anon:0kB inactive_anon:0kB >>> active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB >>> isolated(file):0kB mapped:0kB dirty:0kB writeback:0kB shmem:0kB >>> shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB >>> unstable:0kB all_unreclaimable? no >>> [ 30.370073] Node 1 active_anon:0kB inactive_anon:0kB >>> active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB >>> isolated(file):0kB mapped:0kB dirty:0kB writeback:0kB shmem:0kB >>> shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB >>> unstable:0kB all_unreclaimable? no >>> [ 30.370079] Node 0 DMA free:0kB min:0kB low:0kB high:0kB >>> reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB >>> active_file:0kB inactive_file:0kB unevictable:0kB writepending
Re: [PATCH v8 0/5] support reserving crashkernel above 4G on arm64 kdump
Hi Baoquan, Thanks for your suggestions. You are right, some details should be made in the commit log. Thanks, Chen Zhou On 2020/5/26 9:42, Baoquan He wrote: > On 05/21/20 at 05:38pm, Chen Zhou wrote: >> This patch series enable reserving crashkernel above 4G in arm64. >> >> There are following issues in arm64 kdump: >> 1. We use crashkernel=X to reserve crashkernel below 4G, which will fail >> when there is no enough low memory. >> 2. Currently, crashkernel=Y@X can be used to reserve crashkernel above 4G, >> in this case, if swiotlb or DMA buffers are required, crash dump kernel >> will boot failure because there is no low memory available for allocation. >> >> To solve these issues, introduce crashkernel=X,low to reserve specified >> size low memory. >> Crashkernel=X tries to reserve memory for the crash dump kernel under >> 4G. If crashkernel=Y,low is specified simultaneously, reserve spcified >> size low memory for crash kdump kernel devices firstly and then reserve >> memory above 4G. >> >> When crashkernel is reserved above 4G in memory, that is, crashkernel=X,low >> is specified simultaneously, kernel should reserve specified size low memory >> for crash dump kernel devices. So there may be two crash kernel regions, one >> is below 4G, the other is above 4G. >> In order to distinct from the high region and make no effect to the use of >> kexec-tools, rename the low region as "Crash kernel (low)", and add DT >> property >> "linux,low-memory-range" to crash dump kernel's dtb to pass the low region. >> >> Besides, we need to modify kexec-tools: >> arm64: kdump: add another DT property to crash dump kernel's dtb(see [1]) >> >> The previous changes and discussions can be retrieved from: >> >> Changes since [v7] >> - Move x86 CRASH_ALIGN to 2M >> Suggested by Dave and do some test, move x86 CRASH_ALIGN to 2M. > OK, moving x86 CRASH_ALIGN to 2M is suggested by Dave. Because > CONFIG_PHYSICAL_ALIGN can be selected from 2M to 16M. So 2M seems good. > But, anyway, we should tell the reason why it need be changed in commit > log. > > > arch/x86/Kconfig: > config PHYSICAL_ALIGN > hex "Alignment value to which kernel should be aligned" > default "0x20" > range 0x2000 0x100 if X86_32 > range 0x20 0x100 if X86_64 > >> - Update Documentation/devicetree/bindings/chosen.txt >> Add corresponding documentation to >> Documentation/devicetree/bindings/chosen.txt suggested by Arnd. >> - Add Tested-by from Jhon and pk >> >> Changes since [v6] >> - Fix build errors reported by kbuild test robot. >> >> Changes since [v5] >> - Move reserve_crashkernel_low() into kernel/crash_core.c. >> - Delete crashkernel=X,high. > And the crashkernel=X,high being deleted need be told too. Otherwise > people reading the commit have to check why themselves. I didn't follow > the old version, can't see why ,high can't be specified explicitly. > >> - Modify crashkernel=X,low. >> If crashkernel=X,low is specified simultaneously, reserve spcified size low >> memory for crash kdump kernel devices firstly and then reserve memory above >> 4G. >> In addition, rename crashk_low_res as "Crash kernel (low)" for arm64, and >> then >> pass to crash dump kernel by DT property "linux,low-memory-range". >> - Update Documentation/admin-guide/kdump/kdump.rst. >> >> Changes since [v4] >> - Reimplement memblock_cap_memory_ranges for multiple ranges by Mike. >> >> Changes since [v3] >> - Add memblock_cap_memory_ranges back for multiple ranges. >> - Fix some compiling warnings. >> >> Changes since [v2] >> - Split patch "arm64: kdump: support reserving crashkernel above 4G" as >> two. Put "move reserve_crashkernel_low() into kexec_core.c" in a separate >> patch. >> >> Changes since [v1]: >> - Move common reserve_crashkernel_low() code into kernel/kexec_core.c. >> - Remove memblock_cap_memory_ranges() i added in v1 and implement that >> in fdt_enforce_memory_region(). >> There are at most two crash kernel regions, for two crash kernel regions >> case, we cap the memory range [min(regs[*].start), max(regs[*].end)] >> and then remove the memory range in the middle. >> >> [1]: http://lists.infradead.org/pipermail/kexec/2020-May/025128.html >> [v1]: https://lkml.org/lkml/2019/4/2/1174 >> [v2]: https://lkml.org/lkml/2019/4/9/86 >> [v3]: https://lkml.org/lkml/2019/4/9/306 >> [v4]: https://lkml.org/lkml/2019/4/15/273 >> [v5]: https://lkml.org/lkml/2019/5/6/1360 >> [v6]: https://lkml.org/lkml/2019/8/30/142 >> [v7]: https://lkml.org/lkml/2019/12/23/411 >> >> Chen Zhou (5): >> x86: kdump: move reserve_crashkernel_low() into crash_core.c >> arm64: kdump: reserve crashkenel above 4G for crash dump kernel >> arm64: kdump: add memory for devices by DT property, low-memory-range >> kdump: update Documentation about crashkernel on arm64 >> dt-bindings: chosen: Document linux,low-memory-range for arm64 kdump >> >> Documentation/admin-guide/kdump/kdump.rst | 13 ++- >> .../admin-guide/kernel-parameters.txt
Re: [PATCH v8 5/5] dt-bindings: chosen: Document linux,low-memory-range for arm64 kdump
Hi Rob, On 2020/5/21 21:29, Rob Herring wrote: > On Thu, May 21, 2020 at 3:35 AM Chen Zhou wrote: >> Add documentation for DT property used by arm64 kdump: >> linux,low-memory-range. >> "linux,low-memory-range" is an another memory region used for crash >> dump kernel devices. >> >> Signed-off-by: Chen Zhou >> --- >> Documentation/devicetree/bindings/chosen.txt | 25 >> 1 file changed, 25 insertions(+) > chosen is now a schema documented here[1]. Ok, that is, i don't need to modify the doc in kernel, just create a pull request in github [1]? > >> diff --git a/Documentation/devicetree/bindings/chosen.txt >> b/Documentation/devicetree/bindings/chosen.txt >> index 45e79172a646..bfe6fb6976e6 100644 >> --- a/Documentation/devicetree/bindings/chosen.txt >> +++ b/Documentation/devicetree/bindings/chosen.txt >> @@ -103,6 +103,31 @@ While this property does not represent a real hardware, >> the address >> and the size are expressed in #address-cells and #size-cells, >> respectively, of the root node. >> >> +linux,low-memory-range >> +-- >> +This property (arm64 only) holds a base address and size, describing a >> +limited region below 4G. Similar to "linux,usable-memory-range", it is >> +an another memory range which may be considered available for use by the >> +kernel. > Why can't you just add a range to "linux,usable-memory-range"? It > shouldn't be hard to figure out which part is below 4G. I did like this in my previous version, such as v5. After discussed with James, i modified it to the current way. We think the existing behavior should be unchanged, which helps with keeping compatibility with existing user-space and older kdump kernels. The comments from James: > linux,usable-memory-range = . Won't this break if your kdump kernel doesn't know what the extra parameters are? Or if it expects two ranges, but only gets one? These DT properties should be treated as ABI between kernel versions, we can't really change it like this. I think the 'low' region is an optional-extra, that is never mapped by the first kernel. I think the simplest thing to do is to add an 'linux,low-memory-range' that we memblock_add() after memblock_cap_memory_range() has been called. If its missing, or the new kernel doesn't know what its for, everything keeps working. previous discusses: https://lkml.org/lkml/2019/6/5/674 https://lkml.org/lkml/2019/6/13/229 Thanks, Chen Zhou > > Rob > > [1] > https://github.com/devicetree-org/dt-schema/blob/master/schemas/chosen.yaml > > . > ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v7 0/4] support reserving crashkernel above 4G on arm64 kdump
Hi Arnd, On 2020/5/19 18:21, Arnd Bergmann wrote: > On Thu, Mar 26, 2020 at 4:10 AM Chen Zhou wrote: >> Hi all, >> >> Friendly ping... > I was asked about this patch series, and see that you last posted it in > December. I think you should rebase it to linux-5.7-rc6 and post the > entire series again to make progress, as it's unlikely that any maintainer > would pick up the patches from last year. > > For the contents, everything seems reasonable to me, but I noticed that > you are adding a property to the /chosen node without adding the > corresponding documentation to > Documentation/devicetree/bindings/chosen.txt > > Please add that, and Cc the devicetree maintainers on the updated > patch. > > Arnd Thanks for your review and comments, i will rebase it to linux-5.7-rc6 and add the corresponding documentation. Thanks, Chen Zhou >> On 2019/12/23 23:23, Chen Zhou wrote: >>> This patch series enable reserving crashkernel above 4G in arm64. >>> >>> There are following issues in arm64 kdump: >>> 1. We use crashkernel=X to reserve crashkernel below 4G, which will fail >>> when there is no enough low memory. >>> 2. Currently, crashkernel=Y@X can be used to reserve crashkernel above 4G, >>> in this case, if swiotlb or DMA buffers are required, crash dump kernel >>> will boot failure because there is no low memory available for allocation. >>> >>> The previous changes and discussions can be retrieved from: >>> >>> Changes since [v6] >>> - Fix build errors reported by kbuild test robot. > ... > > . > ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v7 2/4] arm64: kdump: reserve crashkenel above 4G for crash dump kernel
On 2020/3/11 1:08, Prabhakar Kushwaha wrote:
> On 3/10/2020 7:00 AM, chenzhou wrote:
>> Hi,
>>
>> On 2020/3/9 23:51, Prabhakar Kushwaha wrote:
>>> On 3/9/2020 10:18 AM, Prabhakar Kushwaha wrote:
>>>> Hi Chen,
>>>>
>>>> On Sat, Mar 7, 2020 at 4:36 PM Chen Zhou wrote:
>>>>>
>>>>>
>>>>>
>>>>> On 2020/3/5 18:13, Prabhakar Kushwaha wrote:
>>>>>> On Mon, Dec 23, 2019 at 8:57 PM Chen Zhou wrote:
>>>>>>>
>>>>>>> Crashkernel=X tries to reserve memory for the crash dump kernel under
>>>>>>> 4G. If crashkernel=X,low is specified simultaneously, reserve spcified
>>>>>>> size low memory for crash kdump kernel devices firstly and then reserve
>>>>>>> memory above 4G.
>>>>>>>
>>>>>>> Signed-off-by: Chen Zhou
>>>>>>> ---
>>>>>>> arch/arm64/kernel/setup.c | 8 +++-
>>>>>>> arch/arm64/mm/init.c | 31 +--
>>>>>>> 2 files changed, 36 insertions(+), 3 deletions(-)
>>>>>>>
>>>>>>> diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
>>>>>>> index 56f6645..04d1c87 100644
>>>>>>> --- a/arch/arm64/kernel/setup.c
>>>>>>> +++ b/arch/arm64/kernel/setup.c
>>>>>>> @@ -238,7 +238,13 @@ static void __init request_standard_resources(void)
>>>>>>> kernel_data.end <= res->end)
>>>>>>> request_resource(res, &kernel_data);
>>>>>>> #ifdef CONFIG_KEXEC_CORE
>>>>>>> - /* Userspace will find "Crash kernel" region in
>>>>>>> /proc/iomem. */
>>>>>>> + /*
>>>>>>> +* Userspace will find "Crash kernel" region in
>>>>>>> /proc/iomem.
>>>>>>> +* Note: the low region is renamed as Crash kernel
>>>>>>> (low).
>>>>>>> +*/
>>>>>>> + if (crashk_low_res.end && crashk_low_res.start >=
>>>>>>> res->start &&
>>>>>>> + crashk_low_res.end <= res->end)
>>>>>>> + request_resource(res, &crashk_low_res);
>>>>>>> if (crashk_res.end && crashk_res.start >= res->start &&
>>>>>>> crashk_res.end <= res->end)
>>>>>>> request_resource(res, &crashk_res);
>>>>>>> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
>>>>>>> index b65dffd..0d7afd5 100644
>>>>>>> --- a/arch/arm64/mm/init.c
>>>>>>> +++ b/arch/arm64/mm/init.c
>>>>>>> @@ -80,6 +80,7 @@ static void __init reserve_crashkernel(void)
>>>>>>> {
>>>>>>> unsigned long long crash_base, crash_size;
>>>>>>> int ret;
>>>>>>> + phys_addr_t crash_max = arm64_dma32_phys_limit;
>>>>>>>
>>>>>>> ret = parse_crashkernel(boot_command_line,
>>>>>>> memblock_phys_mem_size(),
>>>>>>> &crash_size, &crash_base);
>>>>>>> @@ -87,12 +88,38 @@ static void __init reserve_crashkernel(void)
>>>>>>> if (ret || !crash_size)
>>>>>>> return;
>>>>>>>
>>>>>>> + ret = reserve_crashkernel_low();
>>>>>>> + if (!ret && crashk_low_res.end) {
>>>>>>> + /*
>>>>>>> +* If crashkernel=X,low specified, there may be two
>>>>>>> regions,
>>>>>>> +* we need to make some changes as follows:
>>>>>>> +*
>>>>>>> +* 1. rename the low region as "Crash kernel (low)"
>>>>>>> +* In order to distinct from the high region and make
>>>>>>> no effect
>>>>>>> +* to
Re: [PATCH v7 2/4] arm64: kdump: reserve crashkenel above 4G for crash dump kernel
Hi,
On 2020/3/9 23:51, Prabhakar Kushwaha wrote:
> On 3/9/2020 10:18 AM, Prabhakar Kushwaha wrote:
>> Hi Chen,
>>
>> On Sat, Mar 7, 2020 at 4:36 PM Chen Zhou wrote:
>>>
>>>
>>>
>>> On 2020/3/5 18:13, Prabhakar Kushwaha wrote:
On Mon, Dec 23, 2019 at 8:57 PM Chen Zhou wrote:
>
> Crashkernel=X tries to reserve memory for the crash dump kernel under
> 4G. If crashkernel=X,low is specified simultaneously, reserve spcified
> size low memory for crash kdump kernel devices firstly and then reserve
> memory above 4G.
>
> Signed-off-by: Chen Zhou
> ---
> arch/arm64/kernel/setup.c | 8 +++-
> arch/arm64/mm/init.c | 31 +--
> 2 files changed, 36 insertions(+), 3 deletions(-)
>
> diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
> index 56f6645..04d1c87 100644
> --- a/arch/arm64/kernel/setup.c
> +++ b/arch/arm64/kernel/setup.c
> @@ -238,7 +238,13 @@ static void __init request_standard_resources(void)
> kernel_data.end <= res->end)
> request_resource(res, &kernel_data);
> #ifdef CONFIG_KEXEC_CORE
> - /* Userspace will find "Crash kernel" region in
> /proc/iomem. */
> + /*
> +* Userspace will find "Crash kernel" region in
> /proc/iomem.
> +* Note: the low region is renamed as Crash kernel (low).
> +*/
> + if (crashk_low_res.end && crashk_low_res.start >=
> res->start &&
> + crashk_low_res.end <= res->end)
> + request_resource(res, &crashk_low_res);
> if (crashk_res.end && crashk_res.start >= res->start &&
> crashk_res.end <= res->end)
> request_resource(res, &crashk_res);
> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> index b65dffd..0d7afd5 100644
> --- a/arch/arm64/mm/init.c
> +++ b/arch/arm64/mm/init.c
> @@ -80,6 +80,7 @@ static void __init reserve_crashkernel(void)
> {
> unsigned long long crash_base, crash_size;
> int ret;
> + phys_addr_t crash_max = arm64_dma32_phys_limit;
>
> ret = parse_crashkernel(boot_command_line,
> memblock_phys_mem_size(),
> &crash_size, &crash_base);
> @@ -87,12 +88,38 @@ static void __init reserve_crashkernel(void)
> if (ret || !crash_size)
> return;
>
> + ret = reserve_crashkernel_low();
> + if (!ret && crashk_low_res.end) {
> + /*
> +* If crashkernel=X,low specified, there may be two
> regions,
> +* we need to make some changes as follows:
> +*
> +* 1. rename the low region as "Crash kernel (low)"
> +* In order to distinct from the high region and make no
> effect
> +* to the use of existing kexec-tools, rename the low
> region as
> +* "Crash kernel (low)".
> +*
> +* 2. change the upper bound for crash memory
> +* Set MEMBLOCK_ALLOC_ACCESSIBLE upper bound for crash
> memory.
> +*
> +* 3. mark the low region as "nomap"
> +* The low region is intended to be used for crash dump
> kernel
> +* devices, just mark the low region as "nomap" simply.
> +*/
> + const char *rename = "Crash kernel (low)";
> +
> + crashk_low_res.name = rename;
> + crash_max = MEMBLOCK_ALLOC_ACCESSIBLE;
> + memblock_mark_nomap(crashk_low_res.start,
> + resource_size(&crashk_low_res));
> + }
> +
> crash_size = PAGE_ALIGN(crash_size);
>
> if (crash_base == 0) {
> /* Current arm64 boot protocol requires 2MB alignment */
> - crash_base = memblock_find_in_range(0,
> arm64_dma32_phys_limit,
> - crash_size, SZ_2M);
> + crash_base = memblock_find_in_range(0, crash_max,
> crash_size,
> + SZ_2M);
> if (crash_base == 0) {
> pr_warn("cannot allocate crashkernel
> (size:0x%llx)\n",
> crash_size);
> --
I tested this patch series on ARM64-ThunderX2 with no issue with
bootargs crashkenel=X@Y crashkernel=250M,low
$ dmesg | grep crash
[0.00] crashkernel reserved: 0x000b8120 -
0x000c8120 (4096 MB)
[0.00] Kernel command line:
>
