[Kernel-packages] [Bug 1928405] Re: mkdir/rm/sleep/ls causes kernel 'BUG: unable to handle kernel paging request'

2021-07-12 Thread Launchpad Bug Tracker
[Expired for linux (Ubuntu) because there has been no activity for 60
days.]

** Changed in: linux (Ubuntu)
   Status: Incomplete => Expired

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1928405

Title:
  mkdir/rm/sleep/ls causes kernel 'BUG: unable to handle kernel paging
  request'

Status in linux package in Ubuntu:
  Expired

Bug description:
  I have a network of stock 16.04.2 LTS (Xenial Xerus) servers with
  entirely unmodified "4.4.0-62-generic #83-Ubuntu" kernel running on a
  private network; they run telemetry programs, mostly sh/php out of
  crontab, with very light user interaction for configuration via apache
  and extremely occasional adminstrator ssh access.  They all are on the
  same hardware: same motherboard, same amount of RAM, vary similar very
  small SATA SSD disks.

  A recent fault made us examine the logs, and we see that since 2017
  about half a dozen servers are reporting kernel bugs about once a
  month.

   BUG: unable to handle kernel paging request at 88032fc00062
   CPU: 0 PID: 26071 Comm: mkdir Not tainted 4.4.0-62-generic #83-Ubuntu

  The details vary.  The most common command is mkdir, but also rm,
  head, basename, ls, sleep. (There are every-minute cronjobs sh-scripts
  which run these commands.)

  About half of the logs show tainted (G, D) and have untainted.

  I have found no pattern with time of day, uptime, load (0.16, 0.22,
  0.25 for following report), day of week.

  This is a typical syslog entry, from 2021-01-29; it has the same issue
  in March and May (Comm: mkdir, but tainted G D).

  Jan 29 19:50:17 hostname kernel: [2315584.884470] BUG: unable to handle 
kernel paging request at 88042fc80062
  Jan 29 19:50:17 hostname kernel: [2315584.884500] IP: [] 
__inc_zone_state+0x19/0x60
  Jan 29 19:50:17 hostname kernel: [2315584.884524] PGD 220b067 PUD 0 
  Jan 29 19:50:17 hostname kernel: [2315584.884538] Oops: 0002 [#1] SMP 
  Jan 29 19:50:17 hostname kernel: [2315584.884552] Modules linked in: ppdev 
snd_hda_codec_realtek snd_hda_codec_generic snd_hda_codec_hdmi snd_hda_intel 
coretemp snd_hda_codec serio_raw snd_hda_core snd_hwdep snd_pcm snd_timer snd 
lpc_ich shpchp soundcore parport_pc mac_hid 8250_fintek parport ib_iser rdma_cm 
iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi 
scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov 
async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 
multipath linear psmouse ahci e1000e libahci ptp pps_core video fjes
  Jan 29 19:50:17 hostname kernel: [2315584.884744] CPU: 1 PID: 10730 Comm: 
mkdir Not tainted 4.4.0-62-generic #83-Ubuntu
  Jan 29 19:50:17 hostname kernel: [2315584.884760] Hardware name:  
/PD11TI, BIOS MTCDT10N.85T.0201.2014.1209.1030 12/09/2014
  Jan 29 19:50:17 hostname kernel: [2315584.884779] task: 880034c13fc0 ti: 
8800c8838000 task.ti: 8800c8838000
  Jan 29 19:50:17 hostname kernel: [2315584.884795] RIP: 
0010:[]  [] __inc_zone_state+0x19/0x60
  Jan 29 19:50:17 hostname kernel: [2315584.884816] RSP: :8800c883bc28  
EFLAGS: 00010203
  Jan 29 19:50:17 hostname kernel: [2315584.884842] RAX: 0001 RBX: 
ea000285d540 RCX: 0002
  Jan 29 19:50:17 hostname kernel: [2315584.884878] RDX: 00030062 RSI: 
0021 RDI: ea000285d540
  Jan 29 19:50:17 hostname kernel: [2315584.884915] RBP: 8800c883bc28 R08: 
81cd2dc4 R09: 81cd2db3
  Jan 29 19:50:17 hostname kernel: [2315584.884951] R10:  R11: 
81cd2da2 R12: 88012fff7f80
  Jan 29 19:50:17 hostname kernel: [2315584.884987] R13: 0080 R14: 
ea000285d500 R15: 88012fff77c0
  Jan 29 19:50:17 hostname kernel: [2315584.885027] FS:  7fa2813a1800() 
GS:88012fc8() knlGS:
  Jan 29 19:50:17 hostname kernel: [2315584.885065] CS:  0010 DS:  ES:  
CR0: 80050033
  Jan 29 19:50:17 hostname kernel: [2315584.885088] CR2: 88042fc80062 CR3: 
34f1 CR4: 06e0
  Jan 29 19:50:17 hostname kernel: [2315584.885125] Stack:
  Jan 29 19:50:17 hostname kernel: [2315584.885144]  8800c883bcf0 
811af98d 88012fff96c0 0001df4d6b62
  Jan 29 19:50:17 hostname kernel: [2315584.885186]  880035327a10 
880035327a00 8800c97a7628 8800c97a7628
  Jan 29 19:50:17 hostname kernel: [2315584.885227]  df4d6b62 
880035327a00 88012fff96d0 
  Jan 29 19:50:17 hostname kernel: [2315584.885269] Call Trace:
  Jan 29 19:50:17 hostname kernel: [2315584.885296]  [] 
zone_statistics+0x5d/0xa0
  Jan 29 19:50:17 hostname kernel: [2315584.885324]  [] 
__alloc_pages_nodemask+0x159/0x2a0
  Jan 29 19:50:17 hostname kernel: [2315584.885355]  [] 
alloc_pages_vma+0xbe/0x240
  Jan 29 19:50:17 hostname kernel: [2315584.885383]  [] 

[Kernel-packages] [Bug 1928405] Re: mkdir/rm/sleep/ls causes kernel 'BUG: unable to handle kernel paging request'

2021-05-13 Thread Jonathan L
> Ubuntu 16.04 (xenial) reached end-of-life on April 29, 2021.

It did indeed!

I'll rebuild one or two of the servers with 20.04 LTE and report back.

As it only infrequently manifests, it will take a while to see whether
that fixes the issue.

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1928405

Title:
  mkdir/rm/sleep/ls causes kernel 'BUG: unable to handle kernel paging
  request'

Status in linux package in Ubuntu:
  Incomplete

Bug description:
  I have a network of stock 16.04.2 LTS (Xenial Xerus) servers with
  entirely unmodified "4.4.0-62-generic #83-Ubuntu" kernel running on a
  private network; they run telemetry programs, mostly sh/php out of
  crontab, with very light user interaction for configuration via apache
  and extremely occasional adminstrator ssh access.  They all are on the
  same hardware: same motherboard, same amount of RAM, vary similar very
  small SATA SSD disks.

  A recent fault made us examine the logs, and we see that since 2017
  about half a dozen servers are reporting kernel bugs about once a
  month.

   BUG: unable to handle kernel paging request at 88032fc00062
   CPU: 0 PID: 26071 Comm: mkdir Not tainted 4.4.0-62-generic #83-Ubuntu

  The details vary.  The most common command is mkdir, but also rm,
  head, basename, ls, sleep. (There are every-minute cronjobs sh-scripts
  which run these commands.)

  About half of the logs show tainted (G, D) and have untainted.

  I have found no pattern with time of day, uptime, load (0.16, 0.22,
  0.25 for following report), day of week.

  This is a typical syslog entry, from 2021-01-29; it has the same issue
  in March and May (Comm: mkdir, but tainted G D).

  Jan 29 19:50:17 hostname kernel: [2315584.884470] BUG: unable to handle 
kernel paging request at 88042fc80062
  Jan 29 19:50:17 hostname kernel: [2315584.884500] IP: [] 
__inc_zone_state+0x19/0x60
  Jan 29 19:50:17 hostname kernel: [2315584.884524] PGD 220b067 PUD 0 
  Jan 29 19:50:17 hostname kernel: [2315584.884538] Oops: 0002 [#1] SMP 
  Jan 29 19:50:17 hostname kernel: [2315584.884552] Modules linked in: ppdev 
snd_hda_codec_realtek snd_hda_codec_generic snd_hda_codec_hdmi snd_hda_intel 
coretemp snd_hda_codec serio_raw snd_hda_core snd_hwdep snd_pcm snd_timer snd 
lpc_ich shpchp soundcore parport_pc mac_hid 8250_fintek parport ib_iser rdma_cm 
iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi 
scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov 
async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 
multipath linear psmouse ahci e1000e libahci ptp pps_core video fjes
  Jan 29 19:50:17 hostname kernel: [2315584.884744] CPU: 1 PID: 10730 Comm: 
mkdir Not tainted 4.4.0-62-generic #83-Ubuntu
  Jan 29 19:50:17 hostname kernel: [2315584.884760] Hardware name:  
/PD11TI, BIOS MTCDT10N.85T.0201.2014.1209.1030 12/09/2014
  Jan 29 19:50:17 hostname kernel: [2315584.884779] task: 880034c13fc0 ti: 
8800c8838000 task.ti: 8800c8838000
  Jan 29 19:50:17 hostname kernel: [2315584.884795] RIP: 
0010:[]  [] __inc_zone_state+0x19/0x60
  Jan 29 19:50:17 hostname kernel: [2315584.884816] RSP: :8800c883bc28  
EFLAGS: 00010203
  Jan 29 19:50:17 hostname kernel: [2315584.884842] RAX: 0001 RBX: 
ea000285d540 RCX: 0002
  Jan 29 19:50:17 hostname kernel: [2315584.884878] RDX: 00030062 RSI: 
0021 RDI: ea000285d540
  Jan 29 19:50:17 hostname kernel: [2315584.884915] RBP: 8800c883bc28 R08: 
81cd2dc4 R09: 81cd2db3
  Jan 29 19:50:17 hostname kernel: [2315584.884951] R10:  R11: 
81cd2da2 R12: 88012fff7f80
  Jan 29 19:50:17 hostname kernel: [2315584.884987] R13: 0080 R14: 
ea000285d500 R15: 88012fff77c0
  Jan 29 19:50:17 hostname kernel: [2315584.885027] FS:  7fa2813a1800() 
GS:88012fc8() knlGS:
  Jan 29 19:50:17 hostname kernel: [2315584.885065] CS:  0010 DS:  ES:  
CR0: 80050033
  Jan 29 19:50:17 hostname kernel: [2315584.885088] CR2: 88042fc80062 CR3: 
34f1 CR4: 06e0
  Jan 29 19:50:17 hostname kernel: [2315584.885125] Stack:
  Jan 29 19:50:17 hostname kernel: [2315584.885144]  8800c883bcf0 
811af98d 88012fff96c0 0001df4d6b62
  Jan 29 19:50:17 hostname kernel: [2315584.885186]  880035327a10 
880035327a00 8800c97a7628 8800c97a7628
  Jan 29 19:50:17 hostname kernel: [2315584.885227]  df4d6b62 
880035327a00 88012fff96d0 
  Jan 29 19:50:17 hostname kernel: [2315584.885269] Call Trace:
  Jan 29 19:50:17 hostname kernel: [2315584.885296]  [] 
zone_statistics+0x5d/0xa0
  Jan 29 19:50:17 hostname kernel: [2315584.885324]  [] 
__alloc_pages_nodemask+0x159/0x2a0
  Jan 29 19:50:17 hostname kernel: [2315584.885355]  [] 

[Kernel-packages] [Bug 1928405] Re: mkdir/rm/sleep/ls causes kernel 'BUG: unable to handle kernel paging request'

2021-05-13 Thread Chris Guiver
Thank you for reporting this bug to Ubuntu.

Ubuntu 16.04 (xenial) reached end-of-life on April 29, 2021.

See this document for currently supported Ubuntu releases:
https://wiki.ubuntu.com/Releases

We appreciate that this bug may be old and you might not be interested
in discussing it any more. But if you are then please upgrade to the
latest Ubuntu version and re-test. If you then find the bug is still
present in the newer Ubuntu version, please add a comment here telling
us which new version it is in.

FYI: You're also referring to unpatched and outdated kernels for
xenial/16.04. If you tried a patched kernel the issue may no longer have
existed (fyi: last xenial kernel was 4.4.0.210.216 available using
standard support which has now ended).

** Changed in: linux (Ubuntu)
   Status: New => Incomplete

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1928405

Title:
  mkdir/rm/sleep/ls causes kernel 'BUG: unable to handle kernel paging
  request'

Status in linux package in Ubuntu:
  Incomplete

Bug description:
  I have a network of stock 16.04.2 LTS (Xenial Xerus) servers with
  entirely unmodified "4.4.0-62-generic #83-Ubuntu" kernel running on a
  private network; they run telemetry programs, mostly sh/php out of
  crontab, with very light user interaction for configuration via apache
  and extremely occasional adminstrator ssh access.  They all are on the
  same hardware: same motherboard, same amount of RAM, vary similar very
  small SATA SSD disks.

  A recent fault made us examine the logs, and we see that since 2017
  about half a dozen servers are reporting kernel bugs about once a
  month.

   BUG: unable to handle kernel paging request at 88032fc00062
   CPU: 0 PID: 26071 Comm: mkdir Not tainted 4.4.0-62-generic #83-Ubuntu

  The details vary.  The most common command is mkdir, but also rm,
  head, basename, ls, sleep. (There are every-minute cronjobs sh-scripts
  which run these commands.)

  About half of the logs show tainted (G, D) and have untainted.

  I have found no pattern with time of day, uptime, load (0.16, 0.22,
  0.25 for following report), day of week.

  This is a typical syslog entry, from 2021-01-29; it has the same issue
  in March and May (Comm: mkdir, but tainted G D).

  Jan 29 19:50:17 hostname kernel: [2315584.884470] BUG: unable to handle 
kernel paging request at 88042fc80062
  Jan 29 19:50:17 hostname kernel: [2315584.884500] IP: [] 
__inc_zone_state+0x19/0x60
  Jan 29 19:50:17 hostname kernel: [2315584.884524] PGD 220b067 PUD 0 
  Jan 29 19:50:17 hostname kernel: [2315584.884538] Oops: 0002 [#1] SMP 
  Jan 29 19:50:17 hostname kernel: [2315584.884552] Modules linked in: ppdev 
snd_hda_codec_realtek snd_hda_codec_generic snd_hda_codec_hdmi snd_hda_intel 
coretemp snd_hda_codec serio_raw snd_hda_core snd_hwdep snd_pcm snd_timer snd 
lpc_ich shpchp soundcore parport_pc mac_hid 8250_fintek parport ib_iser rdma_cm 
iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi 
scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov 
async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 
multipath linear psmouse ahci e1000e libahci ptp pps_core video fjes
  Jan 29 19:50:17 hostname kernel: [2315584.884744] CPU: 1 PID: 10730 Comm: 
mkdir Not tainted 4.4.0-62-generic #83-Ubuntu
  Jan 29 19:50:17 hostname kernel: [2315584.884760] Hardware name:  
/PD11TI, BIOS MTCDT10N.85T.0201.2014.1209.1030 12/09/2014
  Jan 29 19:50:17 hostname kernel: [2315584.884779] task: 880034c13fc0 ti: 
8800c8838000 task.ti: 8800c8838000
  Jan 29 19:50:17 hostname kernel: [2315584.884795] RIP: 
0010:[]  [] __inc_zone_state+0x19/0x60
  Jan 29 19:50:17 hostname kernel: [2315584.884816] RSP: :8800c883bc28  
EFLAGS: 00010203
  Jan 29 19:50:17 hostname kernel: [2315584.884842] RAX: 0001 RBX: 
ea000285d540 RCX: 0002
  Jan 29 19:50:17 hostname kernel: [2315584.884878] RDX: 00030062 RSI: 
0021 RDI: ea000285d540
  Jan 29 19:50:17 hostname kernel: [2315584.884915] RBP: 8800c883bc28 R08: 
81cd2dc4 R09: 81cd2db3
  Jan 29 19:50:17 hostname kernel: [2315584.884951] R10:  R11: 
81cd2da2 R12: 88012fff7f80
  Jan 29 19:50:17 hostname kernel: [2315584.884987] R13: 0080 R14: 
ea000285d500 R15: 88012fff77c0
  Jan 29 19:50:17 hostname kernel: [2315584.885027] FS:  7fa2813a1800() 
GS:88012fc8() knlGS:
  Jan 29 19:50:17 hostname kernel: [2315584.885065] CS:  0010 DS:  ES:  
CR0: 80050033
  Jan 29 19:50:17 hostname kernel: [2315584.885088] CR2: 88042fc80062 CR3: 
34f1 CR4: 06e0
  Jan 29 19:50:17 hostname kernel: [2315584.885125] Stack:
  Jan 29 19:50:17 hostname kernel: [2315584.885144]  8800c883bcf0 
811af98d 

[Kernel-packages] [Bug 1928405] Re: mkdir/rm/sleep/ls causes kernel 'BUG: unable to handle kernel paging request'

2021-05-13 Thread Jonathan L
Typo: should be:

About half of the logs show tainted (G, D) and *HALF* untainted.

These computers are rebooted perhaps once a month, typically by power
failure.

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1928405

Title:
  mkdir/rm/sleep/ls causes kernel 'BUG: unable to handle kernel paging
  request'

Status in linux package in Ubuntu:
  New

Bug description:
  I have a network of stock 16.04.2 LTS (Xenial Xerus) servers with
  entirely unmodified "4.4.0-62-generic #83-Ubuntu" kernel running on a
  private network; they run telemetry programs, mostly sh/php out of
  crontab, with very light user interaction for configuration via apache
  and extremely occasional adminstrator ssh access.  They all are on the
  same hardware: same motherboard, same amount of RAM, vary similar very
  small SATA SSD disks.

  A recent fault made us examine the logs, and we see that since 2017
  about half a dozen servers are reporting kernel bugs about once a
  month.

   BUG: unable to handle kernel paging request at 88032fc00062
   CPU: 0 PID: 26071 Comm: mkdir Not tainted 4.4.0-62-generic #83-Ubuntu

  The details vary.  The most common command is mkdir, but also rm,
  head, basename, ls, sleep. (There are every-minute cronjobs sh-scripts
  which run these commands.)

  About half of the logs show tainted (G, D) and have untainted.

  I have found no pattern with time of day, uptime, load (0.16, 0.22,
  0.25 for following report), day of week.

  This is a typical syslog entry, from 2021-01-29; it has the same issue
  in March and May (Comm: mkdir, but tainted G D).

  Jan 29 19:50:17 hostname kernel: [2315584.884470] BUG: unable to handle 
kernel paging request at 88042fc80062
  Jan 29 19:50:17 hostname kernel: [2315584.884500] IP: [] 
__inc_zone_state+0x19/0x60
  Jan 29 19:50:17 hostname kernel: [2315584.884524] PGD 220b067 PUD 0 
  Jan 29 19:50:17 hostname kernel: [2315584.884538] Oops: 0002 [#1] SMP 
  Jan 29 19:50:17 hostname kernel: [2315584.884552] Modules linked in: ppdev 
snd_hda_codec_realtek snd_hda_codec_generic snd_hda_codec_hdmi snd_hda_intel 
coretemp snd_hda_codec serio_raw snd_hda_core snd_hwdep snd_pcm snd_timer snd 
lpc_ich shpchp soundcore parport_pc mac_hid 8250_fintek parport ib_iser rdma_cm 
iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi 
scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov 
async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 
multipath linear psmouse ahci e1000e libahci ptp pps_core video fjes
  Jan 29 19:50:17 hostname kernel: [2315584.884744] CPU: 1 PID: 10730 Comm: 
mkdir Not tainted 4.4.0-62-generic #83-Ubuntu
  Jan 29 19:50:17 hostname kernel: [2315584.884760] Hardware name:  
/PD11TI, BIOS MTCDT10N.85T.0201.2014.1209.1030 12/09/2014
  Jan 29 19:50:17 hostname kernel: [2315584.884779] task: 880034c13fc0 ti: 
8800c8838000 task.ti: 8800c8838000
  Jan 29 19:50:17 hostname kernel: [2315584.884795] RIP: 
0010:[]  [] __inc_zone_state+0x19/0x60
  Jan 29 19:50:17 hostname kernel: [2315584.884816] RSP: :8800c883bc28  
EFLAGS: 00010203
  Jan 29 19:50:17 hostname kernel: [2315584.884842] RAX: 0001 RBX: 
ea000285d540 RCX: 0002
  Jan 29 19:50:17 hostname kernel: [2315584.884878] RDX: 00030062 RSI: 
0021 RDI: ea000285d540
  Jan 29 19:50:17 hostname kernel: [2315584.884915] RBP: 8800c883bc28 R08: 
81cd2dc4 R09: 81cd2db3
  Jan 29 19:50:17 hostname kernel: [2315584.884951] R10:  R11: 
81cd2da2 R12: 88012fff7f80
  Jan 29 19:50:17 hostname kernel: [2315584.884987] R13: 0080 R14: 
ea000285d500 R15: 88012fff77c0
  Jan 29 19:50:17 hostname kernel: [2315584.885027] FS:  7fa2813a1800() 
GS:88012fc8() knlGS:
  Jan 29 19:50:17 hostname kernel: [2315584.885065] CS:  0010 DS:  ES:  
CR0: 80050033
  Jan 29 19:50:17 hostname kernel: [2315584.885088] CR2: 88042fc80062 CR3: 
34f1 CR4: 06e0
  Jan 29 19:50:17 hostname kernel: [2315584.885125] Stack:
  Jan 29 19:50:17 hostname kernel: [2315584.885144]  8800c883bcf0 
811af98d 88012fff96c0 0001df4d6b62
  Jan 29 19:50:17 hostname kernel: [2315584.885186]  880035327a10 
880035327a00 8800c97a7628 8800c97a7628
  Jan 29 19:50:17 hostname kernel: [2315584.885227]  df4d6b62 
880035327a00 88012fff96d0 
  Jan 29 19:50:17 hostname kernel: [2315584.885269] Call Trace:
  Jan 29 19:50:17 hostname kernel: [2315584.885296]  [] 
zone_statistics+0x5d/0xa0
  Jan 29 19:50:17 hostname kernel: [2315584.885324]  [] 
__alloc_pages_nodemask+0x159/0x2a0
  Jan 29 19:50:17 hostname kernel: [2315584.885355]  [] 
alloc_pages_vma+0xbe/0x240
  Jan 29 19:50:17 hostname kernel: [2315584.885383]  [] 

[Kernel-packages] [Bug 1928405] Re: mkdir/rm/sleep/ls causes kernel 'BUG: unable to handle kernel paging request'

2021-05-13 Thread Jonathan L
Apologies for horribly formatted syslog output in description, file
attached.

** Attachment added: "syslog entry of kernel BUG"
   
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1928405/+attachment/5497290/+files/logentry

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1928405

Title:
  mkdir/rm/sleep/ls causes kernel 'BUG: unable to handle kernel paging
  request'

Status in linux package in Ubuntu:
  New

Bug description:
  I have a network of stock 16.04.2 LTS (Xenial Xerus) servers with
  entirely unmodified "4.4.0-62-generic #83-Ubuntu" kernel running on a
  private network; they run telemetry programs, mostly sh/php out of
  crontab, with very light user interaction for configuration via apache
  and extremely occasional adminstrator ssh access.  They all are on the
  same hardware: same motherboard, same amount of RAM, vary similar very
  small SATA SSD disks.

  A recent fault made us examine the logs, and we see that since 2017
  about half a dozen servers are reporting kernel bugs about once a
  month.

   BUG: unable to handle kernel paging request at 88032fc00062
   CPU: 0 PID: 26071 Comm: mkdir Not tainted 4.4.0-62-generic #83-Ubuntu

  The details vary.  The most common command is mkdir, but also rm,
  head, basename, ls, sleep. (There are every-minute cronjobs sh-scripts
  which run these commands.)

  About half of the logs show tainted (G, D) and have untainted.

  I have found no pattern with time of day, uptime, load (0.16, 0.22,
  0.25 for following report), day of week.

  This is a typical syslog entry, from 2021-01-29; it has the same issue
  in March and May (Comm: mkdir, but tainted G D).

  Jan 29 19:50:17 hostname kernel: [2315584.884470] BUG: unable to handle 
kernel paging request at 88042fc80062
  Jan 29 19:50:17 hostname kernel: [2315584.884500] IP: [] 
__inc_zone_state+0x19/0x60
  Jan 29 19:50:17 hostname kernel: [2315584.884524] PGD 220b067 PUD 0 
  Jan 29 19:50:17 hostname kernel: [2315584.884538] Oops: 0002 [#1] SMP 
  Jan 29 19:50:17 hostname kernel: [2315584.884552] Modules linked in: ppdev 
snd_hda_codec_realtek snd_hda_codec_generic snd_hda_codec_hdmi snd_hda_intel 
coretemp snd_hda_codec serio_raw snd_hda_core snd_hwdep snd_pcm snd_timer snd 
lpc_ich shpchp soundcore parport_pc mac_hid 8250_fintek parport ib_iser rdma_cm 
iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi 
scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov 
async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 
multipath linear psmouse ahci e1000e libahci ptp pps_core video fjes
  Jan 29 19:50:17 hostname kernel: [2315584.884744] CPU: 1 PID: 10730 Comm: 
mkdir Not tainted 4.4.0-62-generic #83-Ubuntu
  Jan 29 19:50:17 hostname kernel: [2315584.884760] Hardware name:  
/PD11TI, BIOS MTCDT10N.85T.0201.2014.1209.1030 12/09/2014
  Jan 29 19:50:17 hostname kernel: [2315584.884779] task: 880034c13fc0 ti: 
8800c8838000 task.ti: 8800c8838000
  Jan 29 19:50:17 hostname kernel: [2315584.884795] RIP: 
0010:[]  [] __inc_zone_state+0x19/0x60
  Jan 29 19:50:17 hostname kernel: [2315584.884816] RSP: :8800c883bc28  
EFLAGS: 00010203
  Jan 29 19:50:17 hostname kernel: [2315584.884842] RAX: 0001 RBX: 
ea000285d540 RCX: 0002
  Jan 29 19:50:17 hostname kernel: [2315584.884878] RDX: 00030062 RSI: 
0021 RDI: ea000285d540
  Jan 29 19:50:17 hostname kernel: [2315584.884915] RBP: 8800c883bc28 R08: 
81cd2dc4 R09: 81cd2db3
  Jan 29 19:50:17 hostname kernel: [2315584.884951] R10:  R11: 
81cd2da2 R12: 88012fff7f80
  Jan 29 19:50:17 hostname kernel: [2315584.884987] R13: 0080 R14: 
ea000285d500 R15: 88012fff77c0
  Jan 29 19:50:17 hostname kernel: [2315584.885027] FS:  7fa2813a1800() 
GS:88012fc8() knlGS:
  Jan 29 19:50:17 hostname kernel: [2315584.885065] CS:  0010 DS:  ES:  
CR0: 80050033
  Jan 29 19:50:17 hostname kernel: [2315584.885088] CR2: 88042fc80062 CR3: 
34f1 CR4: 06e0
  Jan 29 19:50:17 hostname kernel: [2315584.885125] Stack:
  Jan 29 19:50:17 hostname kernel: [2315584.885144]  8800c883bcf0 
811af98d 88012fff96c0 0001df4d6b62
  Jan 29 19:50:17 hostname kernel: [2315584.885186]  880035327a10 
880035327a00 8800c97a7628 8800c97a7628
  Jan 29 19:50:17 hostname kernel: [2315584.885227]  df4d6b62 
880035327a00 88012fff96d0 
  Jan 29 19:50:17 hostname kernel: [2315584.885269] Call Trace:
  Jan 29 19:50:17 hostname kernel: [2315584.885296]  [] 
zone_statistics+0x5d/0xa0
  Jan 29 19:50:17 hostname kernel: [2315584.885324]  [] 
__alloc_pages_nodemask+0x159/0x2a0
  Jan 29 19:50:17 hostname kernel: [2315584.885355]  [] 
alloc_pages_vma+0xbe/0x240
  

[Kernel-packages] [Bug 1928405] Re: mkdir/rm/sleep/ls causes kernel 'BUG: unable to handle kernel paging request'

2021-05-13 Thread Jonathan L
result of ubuntu-bug linux

Redacted to remove hostname and private details from syslog

** Attachment added: "Output of ubunut-bug linux"
   
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1928405/+attachment/5497288/+files/REDACTED-apport.linux-image-4.4.0-62-generic.hx5s_1zs.apport

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1928405

Title:
  mkdir/rm/sleep/ls causes kernel 'BUG: unable to handle kernel paging
  request'

Status in linux package in Ubuntu:
  New

Bug description:
  I have a network of stock 16.04.2 LTS (Xenial Xerus) servers with
  entirely unmodified "4.4.0-62-generic #83-Ubuntu" kernel running on a
  private network; they run telemetry programs, mostly sh/php out of
  crontab, with very light user interaction for configuration via apache
  and extremely occasional adminstrator ssh access.  They all are on the
  same hardware: same motherboard, same amount of RAM, vary similar very
  small SATA SSD disks.

  A recent fault made us examine the logs, and we see that since 2017
  about half a dozen servers are reporting kernel bugs about once a
  month.

   BUG: unable to handle kernel paging request at 88032fc00062
   CPU: 0 PID: 26071 Comm: mkdir Not tainted 4.4.0-62-generic #83-Ubuntu

  The details vary.  The most common command is mkdir, but also rm,
  head, basename, ls, sleep. (There are every-minute cronjobs sh-scripts
  which run these commands.)

  About half of the logs show tainted (G, D) and have untainted.

  I have found no pattern with time of day, uptime, load (0.16, 0.22,
  0.25 for following report), day of week.

  This is a typical syslog entry, from 2021-01-29; it has the same issue
  in March and May (Comm: mkdir, but tainted G D).

  Jan 29 19:50:17 hostname kernel: [2315584.884470] BUG: unable to handle 
kernel paging request at 88042fc80062
  Jan 29 19:50:17 hostname kernel: [2315584.884500] IP: [] 
__inc_zone_state+0x19/0x60
  Jan 29 19:50:17 hostname kernel: [2315584.884524] PGD 220b067 PUD 0 
  Jan 29 19:50:17 hostname kernel: [2315584.884538] Oops: 0002 [#1] SMP 
  Jan 29 19:50:17 hostname kernel: [2315584.884552] Modules linked in: ppdev 
snd_hda_codec_realtek snd_hda_codec_generic snd_hda_codec_hdmi snd_hda_intel 
coretemp snd_hda_codec serio_raw snd_hda_core snd_hwdep snd_pcm snd_timer snd 
lpc_ich shpchp soundcore parport_pc mac_hid 8250_fintek parport ib_iser rdma_cm 
iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi 
scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov 
async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 
multipath linear psmouse ahci e1000e libahci ptp pps_core video fjes
  Jan 29 19:50:17 hostname kernel: [2315584.884744] CPU: 1 PID: 10730 Comm: 
mkdir Not tainted 4.4.0-62-generic #83-Ubuntu
  Jan 29 19:50:17 hostname kernel: [2315584.884760] Hardware name:  
/PD11TI, BIOS MTCDT10N.85T.0201.2014.1209.1030 12/09/2014
  Jan 29 19:50:17 hostname kernel: [2315584.884779] task: 880034c13fc0 ti: 
8800c8838000 task.ti: 8800c8838000
  Jan 29 19:50:17 hostname kernel: [2315584.884795] RIP: 
0010:[]  [] __inc_zone_state+0x19/0x60
  Jan 29 19:50:17 hostname kernel: [2315584.884816] RSP: :8800c883bc28  
EFLAGS: 00010203
  Jan 29 19:50:17 hostname kernel: [2315584.884842] RAX: 0001 RBX: 
ea000285d540 RCX: 0002
  Jan 29 19:50:17 hostname kernel: [2315584.884878] RDX: 00030062 RSI: 
0021 RDI: ea000285d540
  Jan 29 19:50:17 hostname kernel: [2315584.884915] RBP: 8800c883bc28 R08: 
81cd2dc4 R09: 81cd2db3
  Jan 29 19:50:17 hostname kernel: [2315584.884951] R10:  R11: 
81cd2da2 R12: 88012fff7f80
  Jan 29 19:50:17 hostname kernel: [2315584.884987] R13: 0080 R14: 
ea000285d500 R15: 88012fff77c0
  Jan 29 19:50:17 hostname kernel: [2315584.885027] FS:  7fa2813a1800() 
GS:88012fc8() knlGS:
  Jan 29 19:50:17 hostname kernel: [2315584.885065] CS:  0010 DS:  ES:  
CR0: 80050033
  Jan 29 19:50:17 hostname kernel: [2315584.885088] CR2: 88042fc80062 CR3: 
34f1 CR4: 06e0
  Jan 29 19:50:17 hostname kernel: [2315584.885125] Stack:
  Jan 29 19:50:17 hostname kernel: [2315584.885144]  8800c883bcf0 
811af98d 88012fff96c0 0001df4d6b62
  Jan 29 19:50:17 hostname kernel: [2315584.885186]  880035327a10 
880035327a00 8800c97a7628 8800c97a7628
  Jan 29 19:50:17 hostname kernel: [2315584.885227]  df4d6b62 
880035327a00 88012fff96d0 
  Jan 29 19:50:17 hostname kernel: [2315584.885269] Call Trace:
  Jan 29 19:50:17 hostname kernel: [2315584.885296]  [] 
zone_statistics+0x5d/0xa0
  Jan 29 19:50:17 hostname kernel: [2315584.885324]  [] 
__alloc_pages_nodemask+0x159/0x2a0
  Jan 29 19:50:17