Re: Fwd: radeon.ko/i586: BUG: kernel NULL pointer dereference, address: 00000004
[TLDR: This mail in primarily relevant for Linux kernel regression tracking. See link in footer if these mails annoy you.] On 14.07.23 04:50, Bagas Sanjaya wrote: > Anyway, I'm adding it to regzbot to ensure it doesn't fall through cracks > unnoticed: > > #regzbot introduced: b39181f7c6907d > https://bugzilla.kernel.org/show_bug.cgi?id=217669 > #regzbot title: FTRACE_MCOUNT_MAX_OFFSET causes kernel NULL pointer > dereference and virtual console (tty1) freeze > #regzbot link: https://gitlab.freedesktop.org/drm/amd/-/issues/2615 #regzbot resolve: root cause likely not solved, but doesn't happen anymore either #regzbot ignore-activity For details see https://gitlab.freedesktop.org/drm/amd/-/issues/2615#note_2108322 Ciao, Thorsten (wearing his 'the Linux kernel's regression tracker' hat) -- Everything you wanna know about Linux kernel regression tracking: https://linux-regtracking.leemhuis.info/about/#tldr That page also explains what to do if mails like this annoy you.
Re: radeon.ko/i586: BUG: kernel NULL pointer dereference,address:00000004
Hi, Thorsten here, the Linux kernel's regression tracker. Top-posting
for once, to make this easily accessible to everyone.
I still have this issue on my list of tracked regressions.
Was this fixed in between? Doesn't look like it from here, but I might
be missing something.
Ciao, Thorsten (wearing his 'the Linux kernel's regression tracker' hat)
--
Everything you wanna know about Linux kernel regression tracking:
https://linux-regtracking.leemhuis.info/about/#tldr
If I did something stupid, please tell me, as explained on that page.
#regzbot poke
On 23.07.23 16:32, Steven Rostedt wrote:
> On Sun, 23 Jul 2023 20:55:06 +0900
> wrote:
>
>> So I tried to trap NULL and return:
>>
>> patch-drm_vblank_cancel_pending_works-printk-NULL-ret.patch
>> diff -up ./drivers/gpu/drm/drm_vblank_work.c.pk2
>> ./drivers/gpu/drm/drm_vblank_work.c
>> --- ./drivers/gpu/drm/drm_vblank_work.c.pk2 2023-06-06 20:50:40.0
>> +0900
>> +++ ./drivers/gpu/drm/drm_vblank_work.c 2023-07-23 14:29:56.383093673
>> +0900
>> @@ -71,6 +71,10 @@ void drm_vblank_cancel_pending_works(str
>> {
>> struct drm_vblank_work *work, *next;
>>
>> +if (!vblank->dev) {
>> +printk(KERN_WARNING "%s: vblank->dev == NULL? returning\n",
>> __func__);
>> +return;
>> +}
>> assert_spin_locked(&vblank->dev->event_lock);
>>
>> list_for_each_entry_safe(work, next, &vblank->pending_work, node) {
>>
>>
>> This time, the printk trap does not happen!! and radeon.ko works.
>> (NULL check for vblank->worker is still fireing though)
>>
>> Now this is puzzling.
>> Is this a timing issue?
>
> It could very well be. And the ftrace patch could possibly not be the
> cause at all. But the thread that is created to do the work is causing
> the race window to be opened up, which is why you see it with the patch
> and don't without it. It may not be the problem, it may just tickle the
> timings enough to trigger the bug, and is causing you to go on a wild
> goose chase in the wrong direction.
>
> -- Steve
>
>
>> Is systemd-udevd doing something not favaorble to kernel?
>> Is drm vblank code running without enough initialization?
>>
>> Puzzling is, that purely useland activity
>> (logging in on tty1 before radeon.ko load)
>> is affecting kernel panic/no-panic.
>
>
>
Re: radeon.ko/i586: BUG: kernel NULL pointer dereference,address:00000004
On Sat, 22 Jul 2023 11:30:14 +0900 wrote: > >> diff --git a/arch/x86/include/asm/ftrace.h b/arch/x86/include/asm/ftrace.h > >> index 897cf02c20b1..801f4414da3e 100644 > >> --- a/arch/x86/include/asm/ftrace.h > >> +++ b/arch/x86/include/asm/ftrace.h > >> @@ -13,7 +13,7 @@ > >> #ifdef CONFIG_HAVE_FENTRY > >> # include > >> /* Add offset for endbr64 if IBT enabled */ > >> -# define FTRACE_MCOUNT_MAX_OFFSET ENDBR_INSN_SIZE > >> +# define FTRACE_MCOUNT_MAX_OFFSET (ENDBR_INSN_SIZE + MCOUNT_INSN_SIZE) > >> #endif > >> > >> #ifdef CONFIG_DYNAMIC_FTRACE > >> > > Above patch didn't work, but > Does it matter that I am compiling with "gcc -fcf-protection=none" > to not emit endbr32 instructions for i586? This patch is supposed to address the case when ENDBR_INSN_SIZE is zero. So I would think that that wouldn't matter. -- Steve
Re: radeon.ko/i586: BUG: kernel NULL pointer dereference, address:00000004
Since the problem with enabling FTRACE_MCOUNT_MAX_OFFSET may be
test_for_valid_rec() returning wrong results, I made a small patch to
catch the result:
patch-test_for_valid_rec-printk.patch
diff -up ./kernel/trace/ftrace.c.ft ./kernel/trace/ftrace.c
--- ./kernel/trace/ftrace.c.ft 2023-07-21 21:51:29.450928552 +0900
+++ ./kernel/trace/ftrace.c 2023-07-23 01:59:59.321558953 +0900
@@ -3678,6 +3678,7 @@ static int test_for_valid_rec(struct dyn
/* Weak functions can cause invalid addresses */
if (!ret || offset > FTRACE_MCOUNT_MAX_OFFSET) {
rec->flags |= FTRACE_FL_DISABLED;
+ printk("%s: disable ftrace for %s offset 0x%lx\n", __func__,
str, offset);
return 0;
}
return 1;
I will attach the console output (with another panic).
Steve, does this look sane?
Another panic seems to occuring here:
void drm_vblank_cancel_pending_works(struct drm_vblank_crtc *vblank)
{
struct drm_vblank_work *work, *next;
assert_spin_locked(&vblank->dev->event_lock); <node);
drm_vblank_put(vblank->dev, vblank->pipe);
}
wake_up_all(&vblank->work_wait_queue);
}
So I tried to trap NULL and return:
patch-drm_vblank_cancel_pending_works-printk-NULL-ret.patch
diff -up ./drivers/gpu/drm/drm_vblank_work.c.pk2
./drivers/gpu/drm/drm_vblank_work.c
--- ./drivers/gpu/drm/drm_vblank_work.c.pk2 2023-06-06 20:50:40.0
+0900
+++ ./drivers/gpu/drm/drm_vblank_work.c 2023-07-23 14:29:56.383093673 +0900
@@ -71,6 +71,10 @@ void drm_vblank_cancel_pending_works(str
{
struct drm_vblank_work *work, *next;
+ if (!vblank->dev) {
+ printk(KERN_WARNING "%s: vblank->dev == NULL? returning\n",
__func__);
+ return;
+ }
assert_spin_locked(&vblank->dev->event_lock);
list_for_each_entry_safe(work, next, &vblank->pending_work, node) {
This time, the printk trap does not happen!! and radeon.ko works.
(NULL check for vblank->worker is still fireing though)
Now this is puzzling.
Is this a timing issue?
Is systemd-udevd doing something not favaorble to kernel?
Is drm vblank code running without enough initialization?
Puzzling is, that purely useland activity
(logging in on tty1 before radeon.ko load)
is affecting kernel panic/no-panic.
--
kabe[0.00] Linux version 5.18.0-13.4.el9.v1.i586 (k...@rocky9.five.ten)
(gcc (GCC) 11.3.1 20221121 (Red Hat 11.3.1-4), GNU ld version 2.35.2-24.el9.v1)
#1 SMP PREEMPT_DYNAMIC Sun Jul 23 11:55:32 JST 2023
[0.00] x86/fpu: x87 FPU will use FSAVE
[0.00] signal: max sigframe size: 928
[0.00] BIOS-provided physical RAM map:
[0.00] BIOS-e820: [mem 0x-0x0009fbff] usable
[0.00] BIOS-e820: [mem 0x0009fc00-0x0009] reserved
[0.00] BIOS-e820: [mem 0x000e-0x000f] reserved
[0.00] BIOS-e820: [mem 0x0010-0x3ffe] usable
[0.00] BIOS-e820: [mem 0x3fff-0x3fff7fff] ACPI data
[0.00] BIOS-e820: [mem 0x3fff8000-0x3fff] ACPI NVS
[0.00] BIOS-e820: [mem 0xfec0-0xfec00fff] reserved
[0.00] BIOS-e820: [mem 0xfee0-0xfee00fff] reserved
[0.00] BIOS-e820: [mem 0xfffe-0x] reserved
[0.00] Notice: NX (Execute Disable) protection missing in CPU!
[0.00] Legacy DMI 2.0 present.
[0.00] DMI: System Manufacturer System Name/ALADDIN5, BIOS 0626 07/15/95
[0.00] tsc: Fast TSC calibration using PIT
[0.00] tsc: Detected 120.091 MHz processor
[0.035691] last_pfn = 0x3fff0 max_arch_pfn = 0x10
[0.035797] Disabled
[0.035867] x86/PAT: MTRRs disabled, skipping PAT initialization too.
[0.035967] x86/PAT: Configuration [0-7]: WB WT UC- UC WB WT UC- UC
[0.114904] RAMDISK: [mem 0x360a5000-0x3704afff]
[0.115027] Allocated new RAMDISK: [mem 0x350ff000-0x360a47e0]
[1.057630] Move RAMDISK from [mem 0x360a5000-0x3704a7e0] to [mem
0x350ff000-0x360a47e0]
[1.057799] ACPI: Early table checksum verification disabled
[1.058084] ACPI: RSDP 0x000F6DF0 14 (v00 AMIINT)
[1.058297] ACPI: RSDT 0x3FFF 28 (v01 ALi_
MSFT 0097)
[1.058576] ACPI: FACP 0x3FFF0030 74 (v01 ALi_
MSFT 0097)
[1.058886] ACPI: DSDT 0x3FFF00B0 001FAA (v01 MSIMSI-5169
1000 MSFT 010A)
[1.059139] ACPI: FACS 0x3FFF8000 40
[1.059335] ACPI: Reserving FACP table memory at [mem 0x3fff0030-0x3fff00a3]
[1.059451] ACPI: Reserving DSDT table memory at [mem 0x3fff00b0-0x3fff2059]
[1.059560] ACPI: Reserving FACS table memory at [mem 0x3fff80
Re: radeon.ko/i586: BUG: kernel NULL pointer dereference,address:00000004
On Sun, 23 Jul 2023 20:55:06 +0900
wrote:
> So I tried to trap NULL and return:
>
> patch-drm_vblank_cancel_pending_works-printk-NULL-ret.patch
> diff -up ./drivers/gpu/drm/drm_vblank_work.c.pk2
> ./drivers/gpu/drm/drm_vblank_work.c
> --- ./drivers/gpu/drm/drm_vblank_work.c.pk2 2023-06-06 20:50:40.0
> +0900
> +++ ./drivers/gpu/drm/drm_vblank_work.c 2023-07-23 14:29:56.383093673
> +0900
> @@ -71,6 +71,10 @@ void drm_vblank_cancel_pending_works(str
> {
> struct drm_vblank_work *work, *next;
>
> + if (!vblank->dev) {
> + printk(KERN_WARNING "%s: vblank->dev == NULL? returning\n",
> __func__);
> + return;
> + }
> assert_spin_locked(&vblank->dev->event_lock);
>
> list_for_each_entry_safe(work, next, &vblank->pending_work, node) {
>
>
> This time, the printk trap does not happen!! and radeon.ko works.
> (NULL check for vblank->worker is still fireing though)
>
> Now this is puzzling.
> Is this a timing issue?
It could very well be. And the ftrace patch could possibly not be the
cause at all. But the thread that is created to do the work is causing
the race window to be opened up, which is why you see it with the patch
and don't without it. It may not be the problem, it may just tickle the
timings enough to trigger the bug, and is causing you to go on a wild
goose chase in the wrong direction.
-- Steve
> Is systemd-udevd doing something not favaorble to kernel?
> Is drm vblank code running without enough initialization?
>
> Puzzling is, that purely useland activity
> (logging in on tty1 before radeon.ko load)
> is affecting kernel panic/no-panic.
Re: radeon.ko/i586: BUG: kernel NULL pointer dereference, address:00000004
rost...@goodmis.org sed in <20230717113623.41878...@gandalf.local.home> >> On Fri, 14 Jul 2023 14:34:04 +0900 >> wrote: >> >> > Patch in >> > https://bugzilla.kernel.org/show_bug.cgi?id=217669#c4 >> > fixed the problem in freedesktop.org kernel 5.18.0-rc2 . >> > This may explain that in kernel.org tree, the said commit is in >> > kernel-5.19. >> >> You mean the patch that adds: >> >> #if defined(FTRACE_MCOUNT_MAX_OFFSET) && (FTRACE_MCOUNT_MAX_OFFSET) >> >> ? >> >> Nothing should be setting FTRACE_MCOUNT_MAX_OFFSET to anything but non >> zero. But doing a grep, I now see: >> >> # define FTRACE_MCOUNT_MAX_OFFSET ENDBR_INSN_SIZE >> >> Where it breaks that assumption if ENDBR_INSN_SIZE == 0 :-p >> (and that's my code!) >> >> OK, does this fix it? (I haven't tested nor compiled this) >> >> -- Steve >> >> diff --git a/arch/x86/include/asm/ftrace.h b/arch/x86/include/asm/ftrace.h >> index 897cf02c20b1..801f4414da3e 100644 >> --- a/arch/x86/include/asm/ftrace.h >> +++ b/arch/x86/include/asm/ftrace.h >> @@ -13,7 +13,7 @@ >> #ifdef CONFIG_HAVE_FENTRY >> # include >> /* Add offset for endbr64 if IBT enabled */ >> -# define FTRACE_MCOUNT_MAX_OFFSET ENDBR_INSN_SIZE >> +# define FTRACE_MCOUNT_MAX_OFFSET (ENDBR_INSN_SIZE + MCOUNT_INSN_SIZE) >> #endif >> >> #ifdef CONFIG_DYNAMIC_FTRACE >> Above patch didn't work, but Does it matter that I am compiling with "gcc -fcf-protection=none" to not emit endbr32 instructions for i586? -- kabe
Re: radeon.ko/i586: BUG: kernel NULL pointer dereference, address:00000004
rost...@goodmis.org sed in <20230717113623.41878...@gandalf.local.home> >> On Fri, 14 Jul 2023 14:34:04 +0900 >> wrote: >> >> > Patch in >> > https://bugzilla.kernel.org/show_bug.cgi?id=217669#c4 >> > fixed the problem in freedesktop.org kernel 5.18.0-rc2 . >> > This may explain that in kernel.org tree, the said commit is in >> > kernel-5.19. >> >> You mean the patch that adds: >> >> #if defined(FTRACE_MCOUNT_MAX_OFFSET) && (FTRACE_MCOUNT_MAX_OFFSET) >> >> ? >> >> Nothing should be setting FTRACE_MCOUNT_MAX_OFFSET to anything but non >> zero. But doing a grep, I now see: >> >> # define FTRACE_MCOUNT_MAX_OFFSET ENDBR_INSN_SIZE >> >> Where it breaks that assumption if ENDBR_INSN_SIZE == 0 :-p >> (and that's my code!) >> >> OK, does this fix it? (I haven't tested nor compiled this) >> >> -- Steve >> >> diff --git a/arch/x86/include/asm/ftrace.h b/arch/x86/include/asm/ftrace.h >> index 897cf02c20b1..801f4414da3e 100644 >> --- a/arch/x86/include/asm/ftrace.h >> +++ b/arch/x86/include/asm/ftrace.h >> @@ -13,7 +13,7 @@ >> #ifdef CONFIG_HAVE_FENTRY >> # include >> /* Add offset for endbr64 if IBT enabled */ >> -# define FTRACE_MCOUNT_MAX_OFFSET ENDBR_INSN_SIZE >> +# define FTRACE_MCOUNT_MAX_OFFSET (ENDBR_INSN_SIZE + MCOUNT_INSN_SIZE) >> #endif >> >> #ifdef CONFIG_DYNAMIC_FTRACE >> Unfortunately this patch didn't fix freedesktop.org 5.18.0-rc2 tree. (vblank->worker == NULL check fires otherwise a panic) Applying this to kernel.org 6.4.3 results in totally different error as in https://bugzilla.kernel.org/show_bug.cgi?id=217669#c0 so there may be multiple regressions I'm chasing. -- kabe
Re: radeon.ko/i586: BUG: kernel NULL pointer dereference, address:00000004
On Fri, 14 Jul 2023 14:34:04 +0900 wrote: > Patch in > https://bugzilla.kernel.org/show_bug.cgi?id=217669#c4 > fixed the problem in freedesktop.org kernel 5.18.0-rc2 . > This may explain that in kernel.org tree, the said commit is in kernel-5.19. You mean the patch that adds: #if defined(FTRACE_MCOUNT_MAX_OFFSET) && (FTRACE_MCOUNT_MAX_OFFSET) ? Nothing should be setting FTRACE_MCOUNT_MAX_OFFSET to anything but non zero. But doing a grep, I now see: # define FTRACE_MCOUNT_MAX_OFFSET ENDBR_INSN_SIZE Where it breaks that assumption if ENDBR_INSN_SIZE == 0 :-p (and that's my code!) OK, does this fix it? (I haven't tested nor compiled this) -- Steve diff --git a/arch/x86/include/asm/ftrace.h b/arch/x86/include/asm/ftrace.h index 897cf02c20b1..801f4414da3e 100644 --- a/arch/x86/include/asm/ftrace.h +++ b/arch/x86/include/asm/ftrace.h @@ -13,7 +13,7 @@ #ifdef CONFIG_HAVE_FENTRY # include /* Add offset for endbr64 if IBT enabled */ -# define FTRACE_MCOUNT_MAX_OFFSET ENDBR_INSN_SIZE +# define FTRACE_MCOUNT_MAX_OFFSET (ENDBR_INSN_SIZE + MCOUNT_INSN_SIZE) #endif #ifdef CONFIG_DYNAMIC_FTRACE
Re: radeon.ko/i586: BUG: kernel NULL pointer dereference,address:00000004
On Sat, 15 Jul 2023 11:39:11 +0900
wrote:
> Yes, this is puzzling. That's why I need other people's opinion on this.
> Does it matter the DUT is a slow machine (Pentium 120MHz)?
>
Hmm, I wonder because the workqueue is running __init functions, could it
possibly be that it didn't finish before the end of boot, where it frees
all the functions? It shouldn't do that because there's code to make sure
it's done before it continues further.
But just in case something isn't working as planned, you could try this
patch to see if the bug goes away.
-- Steve
diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
index 05c0024815bf..af5a40ef3be4 100644
--- a/kernel/trace/ftrace.c
+++ b/kernel/trace/ftrace.c
@@ -3771,13 +3771,13 @@ static int test_for_valid_rec(struct dyn_ftrace *rec)
return 1;
}
-static struct workqueue_struct *ftrace_check_wq __initdata;
-static struct work_struct ftrace_check_work __initdata;
+static struct workqueue_struct *ftrace_check_wq;
+static struct work_struct ftrace_check_work;
/*
* Scan all the mcount/fentry entries to make sure they are valid.
*/
-static __init void ftrace_check_work_func(struct work_struct *work)
+static void ftrace_check_work_func(struct work_struct *work)
{
struct ftrace_page *pg;
struct dyn_ftrace *rec;
Re: radeon.ko/i586: BUG: kernel NULL pointer dereference, address:00000004
rost...@goodmis.org sed in <20230714100019.6bf9b...@gandalf.local.home> >> On Fri, 14 Jul 2023 14:34:04 +0900 >> wrote: >> >> > >> > So I'm confused about why it's mentioned. Was it backported? >> > >> >> > >> Taketo Kabe, could you please help to clean this confusion up? Did you >> > >> mean 5.19 in https://bugzilla.kernel.org/show_bug.cgi?id=217669#c5 ? And >> > >> BTW: did you really use a vanilla kernel for your bisection? >> > >> > >> > Reporter Me: >> > I bisected using freedesktop.org kernel tree, which git commit ID is >> > in sync with kernel.org >> > but version number in ./Makefile could be slighty behind. >> > >> > Patch in >> > https://bugzilla.kernel.org/show_bug.cgi?id=217669#c4 >> > fixed the problem in freedesktop.org kernel 5.18.0-rc2 . >> > This may explain that in kernel.org tree, the said commit is in >> > kernel-5.19. >> >> Even if the bisect did land on this commit, it doesn't make sense. I would >> think that one of the results of the bisect was incorrect (a pass that >> should have failed?), as that would lead the bisect down to the wrong >> conclusion. >> >> Now if you you remove this commit and everything works fine, and add it >> back again and it fails reliably, then I can't argue it is not the commit. I agree with that it does not make sense. But reverting that commit made the freedesktop.org kernel-5.18-0-rc2 not panic, and adding it back made kernel panic (actually, check for vblank->worker==NULL fires) >> >> But the commit in question kicks off a worker thread at boot up to search >> for weak functions that were tagged to be traced by the function tracer and >> sets them to "disabled" to never be traced. >> >> Is the function tracer used at all here? I really do not see how this >> commit affects the code that is crashing. Unless there's something wrong >> with the way the kworker was set up and it corrupted other kworkers :-/ >> -- Steve Yes, this is puzzling. That's why I need other people's opinion on this. Does it matter the DUT is a slow machine (Pentium 120MHz)? -- kabe
Re: radeon.ko/i586: BUG: kernel NULL pointer dereference, address:00000004
On Fri, 14 Jul 2023 14:34:04 +0900 wrote: > >> > So I'm confused about why it's mentioned. Was it backported? > >> > >> Taketo Kabe, could you please help to clean this confusion up? Did you > >> mean 5.19 in https://bugzilla.kernel.org/show_bug.cgi?id=217669#c5 ? And > >> BTW: did you really use a vanilla kernel for your bisection? > > > Reporter Me: > I bisected using freedesktop.org kernel tree, which git commit ID is > in sync with kernel.org > but version number in ./Makefile could be slighty behind. > > Patch in > https://bugzilla.kernel.org/show_bug.cgi?id=217669#c4 > fixed the problem in freedesktop.org kernel 5.18.0-rc2 . > This may explain that in kernel.org tree, the said commit is in kernel-5.19. Even if the bisect did land on this commit, it doesn't make sense. I would think that one of the results of the bisect was incorrect (a pass that should have failed?), as that would lead the bisect down to the wrong conclusion. Now if you you remove this commit and everything works fine, and add it back again and it fails reliably, then I can't argue it is not the commit. But the commit in question kicks off a worker thread at boot up to search for weak functions that were tagged to be traced by the function tracer and sets them to "disabled" to never be traced. Is the function tracer used at all here? I really do not see how this commit affects the code that is crashing. Unless there's something wrong with the way the kworker was set up and it corrupted other kworkers :-/ -- Steve
Re: radeon.ko/i586: BUG: kernel NULL pointer dereference, address: 00000004
On Fri, 14 Jul 2023 09:50:17 +0700 Bagas Sanjaya wrote: > Hi, > > I notice a regression report on Bugzilla [1]. Quoting from it: > > > See Bugzilla for the full thread and attached patches that fixes > this regression. > > Later, when bisecting, the reporter got better kernel trace: > > > [ 469.825305] BUG: kernel NULL pointer dereference, address: 0004 > > [ 469.830502] #PF: supervisor read access in kernel mode > > [ 469.830502] #PF: error_code(0x) - not-present page > > [ 469.830502] *pde = > > [ 469.830502] Oops: [#1] PREEMPT SMP > > [ 469.830502] CPU: 0 PID: 365 Comm: systemd-udevd Not tainted > > 5.14.0-221.el9.v1.i586 #1 This is a 5.14 kernel right? > > [ 469.830502] Hardware name: System Manufacturer System Name/ALADDIN5, > > BIOS 0626 07/15/95 > > [ 469.830502] EIP: _raw_spin_lock_irqsave+0x1f/0x40 > > [ 469.830502] Code: cc cc cc cc cc cc cc 3e cc cc cc 3e 55 89 c1 89 55 89 > > c1 89 5b fa 64 ff 5b fa 64 ff c2 31 d2 be c2 31 d2 be 89 d0 3e 0f 89 d0 > > <3e> 0f 89 d8 5b 5e 89 d8 5b 5e 26 00 90 89 26 00 90 89 b7 15 75 ff > > [ 469.830502] EAX: EBX: 0246 ECX: 0004 EDX: > > [ 469.830502] ESI: 0001 EDI: c3e71c40 EBP: c3e71c34 ESP: c3e71c2c > > [ 469.830502] DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 EFLAGS: 00010046 > > [ 469.830502] CR0: 80050033 CR2: 0004 CR3: 057fa000 CR4: 0010 > > [ 469.830502] Call Trace: > > [ 469.830502] kthread_flush_worker+0x52/0xa0 > > [ 469.830502] ? kthread_should_park+0x40/0x40 > > [ 469.830502] drm_crtc_vblank_off+0x1d7/0x230 [drm] > > [ 469.830502] radeon_crtc_dpms+0x197/0x1b0 [radeon] > > [ 469.830502] radeon_crtc_disable+0x16/0xa0 [radeon] > > [ 469.830502] __drm_helper_disable_unused_functions+0x74/0xc0 > > [drm_kms_helper] > > [ 469.830502] drm_helper_disable_unused_functions+0x3c/0x50 > > [drm_kms_helper] > > [ 469.830502] radeon_fbdev_init+0xb0/0x130 [radeon] > > [ 469.830502] radeon_modeset_init+0x25d/0x320 [radeon] > > [ 469.830502] radeon_driver_load_kms+0xc4/0x240 [radeon] > > [ 469.830502] drm_dev_register+0xb4/0x1a0 [drm] > > [ 469.830502] radeon_pci_probe+0xc0/0x100 [radeon] > > [ 469.830502] pci_device_probe+0xbc/0x150 > > [ 469.830502] really_probe+0xb7/0x350 > > [ 469.830502] __driver_probe_device+0x109/0x1e0 > > [ 469.830502] driver_probe_device+0x1f/0x90 > > [ 469.830502] __driver_attach+0x8a/0x1b0 > > [ 469.830502] ? __device_attach_driver+0x100/0x100 > > [ 469.830502] bus_for_each_dev+0x58/0x90 > > [ 469.830502] driver_attach+0x19/0x20 > > [ 469.830502] ? __device_attach_driver+0x100/0x100 > > [ 469.830502] bus_add_driver+0x12f/0x1d0 > > [ 469.830502] driver_register+0x79/0xd0 > > [ 469.830502] ? 0xf7dde000 > > [ 469.830502] __pci_register_driver+0x52/0x60 > > [ 469.830502] radeon_module_init+0x5c/0x1000 [radeon] > > [ 469.830502] do_one_initcall+0x3e/0x1c0 > > [ 469.830502] ? __vunmap+0x20b/0x2a0 > > [ 469.830502] ? __vunmap+0x20b/0x2a0 > > [ 469.830502] ? kmem_cache_alloc_trace+0x38/0x440 > > [ 469.830502] do_init_module+0x52/0x260 > > [ 469.830502] load_module+0x930/0x9b0 > > [ 469.830502] __ia32_sys_init_module+0x15d/0x180 > > [ 469.830502] do_int80_syscall_32+0x2e/0x80 > > [ 469.830502] entry_INT80_32+0xf0/0xf0 > > [ 469.830502] EIP: 0xb79e7e4e > > [ 469.830502] Code: 0f 83 d6 06 00 00 c3 66 90 66 90 90 57 56 53 8b 7c 24 > > 20 8b 74 24 1c 8b 54 24 18 8b 4c 24 14 8b 5c 24 10 b8 80 00 00 00 cd 80 > > <5b> 5e 5f 3d 01 f0 ff ff 0f 83 a4 06 00 00 c3 66 90 90 53 8b 54 24 > > [ 469.830502] EAX: ffda EBX: b5526010 ECX: 0020d79c EDX: b7c26274 > > [ 469.830502] ESI: b7c20295 EDI: b7c2ddd8 EBP: 018af7c0 ESP: bfd2f810 > > [ 469.830502] DS: 007b ES: 007b FS: GS: 0033 SS: 007b EFLAGS: 0292 > > [ 469.830502] Modules linked in: nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 > > nft_fib radeon(+) nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject > > nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 > > gpu_sched drm_buddy i2c_algo_bit drm_display_helper cec drm_ttm_helper > > ppdev ttm rfkill ip_set nf_tables libcrc32c nfnetlink drm_kms_helper pcspkr > > syscopyarea e100 sysfillrect parport_pc sysimgblt mii fb_sys_fops parport > > qrtr drm fuse ext4 mbcache jbd2 sd_mod t10_pi sr_mod crc64_rocksoft_generic > > cdrom crc64_rocksoft crc64 sg ata_generic pata_ali libata serio_raw > > [ 469.830502] CR2: 0004 > > [ 469.830502] ---[ end trace 30555bd5ee4bee23 ]--- > > [ 469.830502] EIP: _raw_spin_lock_irqsave+0x1f/0x40 > > [ 469.830502] Code: cc cc cc cc cc cc cc 3e cc cc cc 3e 55 89 c1 89 55 89 > > c1 89 5b fa 64 ff 5b fa 64 ff c2 31 d2 be c2 31 d2 be 89 d0 3e 0f 89 d0 > > <3e> 0f 89 d8 5b 5e 89 d8 5b 5e 26 00 90 89 26 00 90 89 b7 15 75 ff > > [ 469.830502] EAX: EBX: 0246 ECX: 0004 EDX: > > [ 469.830502] ESI: 0001 EDI: c3e71c40 EBP: c3e71c34 ESP: c3e71c2c > > [ 469.830502] DS: 007b ES: 007b FS: 00d8 GS: 0033 SS:
Re: radeon.ko/i586: BUG: kernel NULL pointer dereference, address:00000004
Thanks you all for getting attention to the report:
regressi...@leemhuis.info sed in
<55a3bbb1-5b3c-f454-b529-8ee9944cc...@leemhuis.info>
>> On 14.07.23 05:12, Steven Rostedt wrote:
>> > On Fri, 14 Jul 2023 09:50:17 +0700
>> > Bagas Sanjaya wrote:
>> >
>> >> I notice a regression report on Bugzilla [1]. Quoting from it:
>> >>
>> >>
>> >> See Bugzilla for the full thread and attached patches that fixes
>> >> this regression.
>> >>
>> >> Later, when bisecting, the reporter got better kernel trace:
>> >>
>> >>> [ 469.825305] BUG: kernel NULL pointer dereference, address: 0004
>> >>> [ 469.830502] #PF: supervisor read access in kernel mode
>> >>> [ 469.830502] #PF: error_code(0x) - not-present page
>> >>> [ 469.830502] *pde =
>> >>> [ 469.830502] Oops: [#1] PREEMPT SMP
>> >>> [ 469.830502] CPU: 0 PID: 365 Comm: systemd-udevd Not tainted
>> >>> 5.14.0-221.el9.v1.i586 #1
>> >
>> > This is a 5.14 kernel right?
>>
>> And a vendor kernel that from the sound of the version number might be
>> heavily patched. But apparently the reporter later bisected this on a
>> newer kernel (Bagas, would have been good if this had been mentioned in
>> your earlier mail):
>>
>> https://bugzilla.kernel.org/show_bug.cgi?id=217669#c5
>> ```
>> I succeeded to bisect down the regressing commit found in kernel-5.18.0-rc2:
>>
>> b39181f7c690 (refs/bisect/bad) ftrace: Add FTRACE_MCOUNT_MAX_OFFSET to
>> avoid adding weak function
>>
>> This at a glance does not relate to drm/kms code.
>>
>> The attached patch effectively reverts the commit for 32bit.
>> This fixed the problem on kernel-5.18.0, but not enough for kernel-6.4.3 ```
>>
>> That being said: That commit is not in 5.18, as Steve noticed:
>>
>> >> #regzbot introduced: b39181f7c6907d
>> >> https://bugzilla.kernel.org/show_bug.cgi?id=217669
>> >> #regzbot title: FTRACE_MCOUNT_MAX_OFFSET causes kernel NULL pointer
>> >> dereference and virtual console (tty1) freeze
>> > That commit was added in 5.19.
>> >
>> > So I'm confused about why it's mentioned. Was it backported?
>>
>> Taketo Kabe, could you please help to clean this confusion up? Did you
>> mean 5.19 in https://bugzilla.kernel.org/show_bug.cgi?id=217669#c5 ? And
>> BTW: did you really use a vanilla kernel for your bisection?
Reporter Me:
I bisected using freedesktop.org kernel tree, which git commit ID is
in sync with kernel.org
but version number in ./Makefile could be slighty behind.
Patch in
https://bugzilla.kernel.org/show_bug.cgi?id=217669#c4
fixed the problem in freedesktop.org kernel 5.18.0-rc2 .
This may explain that in kernel.org tree, the said commit is in kernel-5.19.
>> TWIMC, there is also
>> https://bugzilla.kernel.org/show_bug.cgi?id=217669#c6 :
>> ```
>> Attached patch sort of fixes the problem; it does not panic and
>> KMS console works, but printk is triggered 4 times on radeon.ko load and
>> when VGA connector is plugged in.
>>
>> I am sort of at loss now; I need advice from people which knows better.
>>
>> --- ./drivers/gpu/drm/drm_internal.h.rd 2023-06-25 21:35:27.506967450
>> +0900
>> +++ ./drivers/gpu/drm/drm_internal.h.rd 2023-06-25 21:36:34.758055363
>> +0900
>> @@ -99,6 +99,10 @@ u64 drm_vblank_count(struct drm_device *
>> /* drm_vblank_work.c */
>> static inline void drm_vblank_flush_worker(struct drm_vblank_crtc *vblank)
>> {
>> + if (!vblank->worker) {
>> + printk(KERN_WARNING "%s: vblank->worker NULL? returning\n",
>> __func__);
>> + return;
>> + }
>> kthread_flush_worker(vblank->worker);
>> }
>> ```
>>
>> Ciao, Thorsten
>>
Fwd: radeon.ko/i586: BUG: kernel NULL pointer dereference, address: 00000004
Hi, I notice a regression report on Bugzilla [1]. Quoting from it: > Origin report: https://gitlab.freedesktop.org/drm/amd/-/issues/2615 > > Compile with CONFIG_DRM_RADEON=m on 32bit, > kernel panics on radeon.ko load. > > On slow machine, there is 70 second window between > login: prompt on raw VGA tty1 and > radeon.ko KMS console load. > > When logging in on tty1 during this window, kernel panics. > (Fast machine loads KMS console before login: prompt, so the problem is > masked) > > [ 466.298267] [drm] radeon kernel modesetting enabled. > [ 466.371716] radeon :01:00.0: vgaarb: deactivate vga console > [ 466.476920] Console: switching to colour dummy device 80x25 > [ 466.661163] [drm] initializing kernel modesetting (RV280 0x1002:0x5960 > 0x148C:0x2094 0x01). > [ 466.672175] [drm] Forcing AGP to PCI mode > [ 466.848372] [drm] Generation 2 PCI interface, using max accessible memory > [ 466.859389] radeon :01:00.0: VRAM: 128M 0xC000 - > 0xC7FF (128M used) > [ 466.869191] radeon :01:00.0: GTT: 512M 0xA000 - > 0xBFFF > [ 467.022337] [drm] Detected VRAM RAM=128M, BAR=128M > [ 467.041498] [drm] RAM width 64bits DDR > [ 467.153587] workqueue: Failed to create a rescuer kthread for wq "ttm": > -EINTR > [ 467.154126] [drm:radeon_ttm_init [radeon]] *ERROR* failed initializing > buffer object driver(-12). > [ 467.176714] radeon :01:00.0: Fatal error during GPU init > [ 467.361714] [drm] radeon: finishing device. > [ 467.431560] [drm] radeon: cp finalized > [ 467.495402] radeon: probe of :01:00.0 failed with error -12 See Bugzilla for the full thread and attached patches that fixes this regression. Later, when bisecting, the reporter got better kernel trace: > [ 469.825305] BUG: kernel NULL pointer dereference, address: 0004 > [ 469.830502] #PF: supervisor read access in kernel mode > [ 469.830502] #PF: error_code(0x) - not-present page > [ 469.830502] *pde = > [ 469.830502] Oops: [#1] PREEMPT SMP > [ 469.830502] CPU: 0 PID: 365 Comm: systemd-udevd Not tainted > 5.14.0-221.el9.v1.i586 #1 > [ 469.830502] Hardware name: System Manufacturer System Name/ALADDIN5, BIOS > 0626 07/15/95 > [ 469.830502] EIP: _raw_spin_lock_irqsave+0x1f/0x40 > [ 469.830502] Code: cc cc cc cc cc cc cc 3e cc cc cc 3e 55 89 c1 89 55 89 c1 > 89 5b fa 64 ff 5b fa 64 ff c2 31 d2 be c2 31 d2 be 89 d0 3e 0f 89 d0 <3e> 0f > 89 d8 5b 5e 89 d8 5b 5e 26 00 90 89 26 00 90 89 b7 15 75 ff > [ 469.830502] EAX: EBX: 0246 ECX: 0004 EDX: > [ 469.830502] ESI: 0001 EDI: c3e71c40 EBP: c3e71c34 ESP: c3e71c2c > [ 469.830502] DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 EFLAGS: 00010046 > [ 469.830502] CR0: 80050033 CR2: 0004 CR3: 057fa000 CR4: 0010 > [ 469.830502] Call Trace: > [ 469.830502] kthread_flush_worker+0x52/0xa0 > [ 469.830502] ? kthread_should_park+0x40/0x40 > [ 469.830502] drm_crtc_vblank_off+0x1d7/0x230 [drm] > [ 469.830502] radeon_crtc_dpms+0x197/0x1b0 [radeon] > [ 469.830502] radeon_crtc_disable+0x16/0xa0 [radeon] > [ 469.830502] __drm_helper_disable_unused_functions+0x74/0xc0 > [drm_kms_helper] > [ 469.830502] drm_helper_disable_unused_functions+0x3c/0x50 [drm_kms_helper] > [ 469.830502] radeon_fbdev_init+0xb0/0x130 [radeon] > [ 469.830502] radeon_modeset_init+0x25d/0x320 [radeon] > [ 469.830502] radeon_driver_load_kms+0xc4/0x240 [radeon] > [ 469.830502] drm_dev_register+0xb4/0x1a0 [drm] > [ 469.830502] radeon_pci_probe+0xc0/0x100 [radeon] > [ 469.830502] pci_device_probe+0xbc/0x150 > [ 469.830502] really_probe+0xb7/0x350 > [ 469.830502] __driver_probe_device+0x109/0x1e0 > [ 469.830502] driver_probe_device+0x1f/0x90 > [ 469.830502] __driver_attach+0x8a/0x1b0 > [ 469.830502] ? __device_attach_driver+0x100/0x100 > [ 469.830502] bus_for_each_dev+0x58/0x90 > [ 469.830502] driver_attach+0x19/0x20 > [ 469.830502] ? __device_attach_driver+0x100/0x100 > [ 469.830502] bus_add_driver+0x12f/0x1d0 > [ 469.830502] driver_register+0x79/0xd0 > [ 469.830502] ? 0xf7dde000 > [ 469.830502] __pci_register_driver+0x52/0x60 > [ 469.830502] radeon_module_init+0x5c/0x1000 [radeon] > [ 469.830502] do_one_initcall+0x3e/0x1c0 > [ 469.830502] ? __vunmap+0x20b/0x2a0 > [ 469.830502] ? __vunmap+0x20b/0x2a0 > [ 469.830502] ? kmem_cache_alloc_trace+0x38/0x440 > [ 469.830502] do_init_module+0x52/0x260 > [ 469.830502] load_module+0x930/0x9b0 > [ 469.830502] __ia32_sys_init_module+0x15d/0x180 > [ 469.830502] do_int80_syscall_32+0x2e/0x80 > [ 469.830502] entry_INT80_32+0xf0/0xf0 > [ 469.830502] EIP: 0xb79e7e4e > [ 469.830502] Code: 0f 83 d6 06 00 00 c3 66 90 66 90 90 57 56 53 8b 7c 24 20 > 8b 74 24 1c 8b 54 24 18 8b 4c 24 14 8b 5c 24 10 b8 80 00 00 00 cd 80 <5b> 5e > 5f 3d 01 f0 ff ff 0f 83 a4 06 00 00 c3 66 90 90 53 8b 54 24 > [ 469.830502] EAX: ffda EBX: b5526010 ECX: 0020d79c EDX: b7c26274 > [ 469.830502] ESI: b7c20295 EDI: b7c2d
Re: radeon.ko/i586: BUG: kernel NULL pointer dereference, address:00000004
Thanks you all for getting attention to the report:
regressi...@leemhuis.info sed in
<55a3bbb1-5b3c-f454-b529-8ee9944cc...@leemhuis.info>
>> On 14.07.23 05:12, Steven Rostedt wrote:
>> > On Fri, 14 Jul 2023 09:50:17 +0700
>> > Bagas Sanjaya wrote:
>> >
>> >> I notice a regression report on Bugzilla [1]. Quoting from it:
>> >>
>> >>
>> >> See Bugzilla for the full thread and attached patches that fixes
>> >> this regression.
>> >>
>> >> Later, when bisecting, the reporter got better kernel trace:
>> >>
>> >>> [ 469.825305] BUG: kernel NULL pointer dereference, address: 0004
>> >>> [ 469.830502] #PF: supervisor read access in kernel mode
>> >>> [ 469.830502] #PF: error_code(0x) - not-present page
>> >>> [ 469.830502] *pde =
>> >>> [ 469.830502] Oops: [#1] PREEMPT SMP
>> >>> [ 469.830502] CPU: 0 PID: 365 Comm: systemd-udevd Not tainted
>> >>> 5.14.0-221.el9.v1.i586 #1
>> >
>> > This is a 5.14 kernel right?
>>
>> And a vendor kernel that from the sound of the version number might be
>> heavily patched. But apparently the reporter later bisected this on a
>> newer kernel (Bagas, would have been good if this had been mentioned in
>> your earlier mail):
>>
>> https://bugzilla.kernel.org/show_bug.cgi?id=217669#c5
>> ```
>> I succeeded to bisect down the regressing commit found in kernel-5.18.0-rc2:
>>
>> b39181f7c690 (refs/bisect/bad) ftrace: Add FTRACE_MCOUNT_MAX_OFFSET to
>> avoid adding weak function
>>
>> This at a glance does not relate to drm/kms code.
>>
>> The attached patch effectively reverts the commit for 32bit.
>> This fixed the problem on kernel-5.18.0, but not enough for kernel-6.4.3 ```
>>
>> That being said: That commit is not in 5.18, as Steve noticed:
>>
>> >> #regzbot introduced: b39181f7c6907d
>> >> https://bugzilla.kernel.org/show_bug.cgi?id=217669
>> >> #regzbot title: FTRACE_MCOUNT_MAX_OFFSET causes kernel NULL pointer
>> >> dereference and virtual console (tty1) freeze
>> > That commit was added in 5.19.
>> >
>> > So I'm confused about why it's mentioned. Was it backported?
>>
>> Taketo Kabe, could you please help to clean this confusion up? Did you
>> mean 5.19 in https://bugzilla.kernel.org/show_bug.cgi?id=217669#c5 ? And
>> BTW: did you really use a vanilla kernel for your bisection?
Reporter Me:
I bisected using freedesktop.org kernel tree, which git commit ID is
in sync with kernel.org
but version number in ./Makefile could be slighty behind.
Patch in
https://bugzilla.kernel.org/show_bug.cgi?id=217669#c4
fixed the problem in freedesktop.org kernel 5.18.0-rc2 .
This may explain that in kernel.org tree, the said commit is in kernel-5.19.
>> TWIMC, there is also
>> https://bugzilla.kernel.org/show_bug.cgi?id=217669#c6 :
>> ```
>> Attached patch sort of fixes the problem; it does not panic and
>> KMS console works, but printk is triggered 4 times on radeon.ko load and
>> when VGA connector is plugged in.
>>
>> I am sort of at loss now; I need advice from people which knows better.
>>
>> --- ./drivers/gpu/drm/drm_internal.h.rd 2023-06-25 21:35:27.506967450
>> +0900
>> +++ ./drivers/gpu/drm/drm_internal.h.rd 2023-06-25 21:36:34.758055363
>> +0900
>> @@ -99,6 +99,10 @@ u64 drm_vblank_count(struct drm_device *
>> /* drm_vblank_work.c */
>> static inline void drm_vblank_flush_worker(struct drm_vblank_crtc *vblank)
>> {
>> + if (!vblank->worker) {
>> + printk(KERN_WARNING "%s: vblank->worker NULL? returning\n",
>> __func__);
>> + return;
>> + }
>> kthread_flush_worker(vblank->worker);
>> }
>> ```
>>
>> Ciao, Thorsten
>>
Re: radeon.ko/i586: BUG: kernel NULL pointer dereference, address: 00000004
On 14.07.23 05:12, Steven Rostedt wrote:
> On Fri, 14 Jul 2023 09:50:17 +0700
> Bagas Sanjaya wrote:
>
>> I notice a regression report on Bugzilla [1]. Quoting from it:
>>
>>
>> See Bugzilla for the full thread and attached patches that fixes
>> this regression.
>>
>> Later, when bisecting, the reporter got better kernel trace:
>>
>>> [ 469.825305] BUG: kernel NULL pointer dereference, address: 0004
>>> [ 469.830502] #PF: supervisor read access in kernel mode
>>> [ 469.830502] #PF: error_code(0x) - not-present page
>>> [ 469.830502] *pde =
>>> [ 469.830502] Oops: [#1] PREEMPT SMP
>>> [ 469.830502] CPU: 0 PID: 365 Comm: systemd-udevd Not tainted
>>> 5.14.0-221.el9.v1.i586 #1
>
> This is a 5.14 kernel right?
And a vendor kernel that from the sound of the version number might be
heavily patched. But apparently the reporter later bisected this on a
newer kernel (Bagas, would have been good if this had been mentioned in
your earlier mail):
https://bugzilla.kernel.org/show_bug.cgi?id=217669#c5
```
I succeeded to bisect down the regressing commit found in kernel-5.18.0-rc2:
b39181f7c690 (refs/bisect/bad) ftrace: Add FTRACE_MCOUNT_MAX_OFFSET to
avoid adding weak function
This at a glance does not relate to drm/kms code.
The attached patch effectively reverts the commit for 32bit.
This fixed the problem on kernel-5.18.0, but not enough for kernel-6.4.3 ```
That being said: That commit is not in 5.18, as Steve noticed:
>> #regzbot introduced: b39181f7c6907d
>> https://bugzilla.kernel.org/show_bug.cgi?id=217669
>> #regzbot title: FTRACE_MCOUNT_MAX_OFFSET causes kernel NULL pointer
>> dereference and virtual console (tty1) freeze
> That commit was added in 5.19.
>
> So I'm confused about why it's mentioned. Was it backported?
Taketo Kabe, could you please help to clean this confusion up? Did you
mean 5.19 in https://bugzilla.kernel.org/show_bug.cgi?id=217669#c5 ? And
BTW: did you really use a vanilla kernel for your bisection?
TWIMC, there is also
https://bugzilla.kernel.org/show_bug.cgi?id=217669#c6 :
```
Attached patch sort of fixes the problem; it does not panic and
KMS console works, but printk is triggered 4 times on radeon.ko load and
when VGA connector is plugged in.
I am sort of at loss now; I need advice from people which knows better.
--- ./drivers/gpu/drm/drm_internal.h.rd2023-06-25 21:35:27.506967450
+0900
+++ ./drivers/gpu/drm/drm_internal.h.rd2023-06-25 21:36:34.758055363
+0900
@@ -99,6 +99,10 @@ u64 drm_vblank_count(struct drm_device *
/* drm_vblank_work.c */
static inline void drm_vblank_flush_worker(struct drm_vblank_crtc *vblank)
{
+ if (!vblank->worker) {
+ printk(KERN_WARNING "%s: vblank->worker NULL? returning\n",
__func__);
+ return;
+ }
kthread_flush_worker(vblank->worker);
}
```
Ciao, Thorsten
