Hi Chengen!
Thank you very much for the detailed test plan! It's really good.
Apologies for the extended delay in getting back to your changes.
There are some points to address in the debdiffs. In order of
importance:
1) The debdiff for Lunar and Jammy (equivalent) are for kernel 6.2, but
Jammy now has the 6.5 HWE kernel [1] too.
I suspect that the users/original reporter that looked for HWE kernel
support back at 6.2 would likely switch to 6.5 too, so it would be
beneficial and time saving to incorporate the 6.5 patches now, since you
already have them in the Mantic debdiff [2]. What do you think?
2) There are 12 patches from crash 8.0.4 [3] in the Mantic debdiff, but
these are not yet in Ubuntu (not even in Noble, the development
release), which has 8.0.3.
I will provide another Merge from Debian to crash, since upstream has now
released 8.0.4 and Debian pulled it in.
But for the future it's important to keep this in mind, as it may save the time
of a review round trip, and more debdiff/build/test work.
3) The version numbers are incorrect for stable releases.
Please see [4].
In general, you should increment by 'ubuntu0.1' in stable releases (and
'ubuntu1' in the development release), and if multiple stable releases
have the same package version, increment by 'ubuntu0.YY.MM.1' to
differentiate among them.
i.e.,
$ rmadison -a source crash
...
crash | 8.0.0-1ubuntu1 | jammy | source
crash | 8.0.0-1ubuntu1 | lunar | source
crash | 8.0.2-1ubuntu1 | mantic | source
crash | 8.0.3+ds1-3ubuntu1 | noble | source
debdiffs:
$ grep -o '.crash (.*)' *.debdiff
lp2038249-crash-jammy.debdiff:+crash (8.0.0-1ubuntu2)
lp2038249-crash-jammy.debdiff: crash (8.0.0-1ubuntu1)
lp2038249-crash-lunar.debdiff:+crash (8.0.0-1ubuntu2)
lp2038249-crash-lunar.debdiff: crash (8.0.0-1ubuntu1)
lp2038249-crash-mantic.debdiff:+crash (8.0.2-1ubuntu2)
lp2038249-crash-mantic.debdiff: crash (8.0.2-1ubuntu1)
per [4]:
jammy: 8.0.0-1ubuntu1.22.04.1
lunar: 8.0.0-1ubuntu1.23.04.1
mantic: 8.0.2-1ubuntu1.1
4) The changes to 'debian/Makefile.ori' are not listed or explained
in the changelog.
If this is unintended (i.e., post-build noise in debian/, it should
be removed), but if it's intended/required, then it should be listed
and slightly documented, please.
5) The itemization in changelog is a bit weird.
This is a style/cosmetic suggestion, not a requirement, but in practice,
when you have multiple patches, the first item is the description with
the LP bug number, and the sub items are the patches; e.g.,
+ * Fix the dump file parsing issue arises from structural changes in Linux
+ kernel 6.5 (LP: #2038249)
+ - d/p/lp2038249-arm64-handle-vabits_actual-symbol-missing-case.patch
+ -
d/p/lp2038249-x86_64-Fix-for-move-of-per-cpu-variables-into-struct.patch
+ - ... more patches
6) The patches are not numbered
Again, a style/cosmetic suggestion; but it does help with parsing/organization
over time for longer patch series like this one.
You can use 'quilt rename' and a shell loop to easily adjust this, of course.
Please let me know if you would like some help (I don't think so, but just in
case!).
7) Special thanks for paying attention to 'upstream/backport' keyword in
the 'Origin:' DEP-3 header!
$ grep -ho 'Origin: [^ ]*' *.debdiff | sort | uniq -c
12 Origin: backport,
66 Origin: upstream,
...
So, for now, if you have a chance, please feel free to adjust the debdiffs,
and I'll work on merging 8.0.4 in Noble, to unblock the needed 12 patches.
Thanks again,
Mauricio
[1] https://launchpad.net/ubuntu/+source/linux-hwe-6.5
[2]
$ diff -U0 \
<(filterdiff --remove-timestamps lp2038249-crash-mantic.debdiff -i
'*/debian/changelog') \
<(filterdiff --remove-timestamps lp2038249-crash-lunar.debdiff -i
'*/debian/changelog')
...
@@ -18,7 +29,0 @@
-+
d/p/lp2038249-Fix-kmem-n-option-to-display-memory-blocks-on-Linux-.patch
-+
d/p/lp2038249-Fix-failure-of-dev-d-D-options-on-Linux-6.4-and-late.patch
-+
d/p/lp2038249-Fix-kmem-v-option-displaying-no-regions-on-Linux-6.3.patch
-+
d/p/lp2038249-x86_64-Fix-bt-command-printing-stale-entries-on-Linu.patch
-+
d/p/lp2038249-Support-module-memory-layout-change-on-Linux-6.4.patch
-+
d/p/lp2038249-Fix-failure-of-gathering-task-table-on-Linux-6.5-rc1.patch
-+
d/p/lp2038249-Fix-compilation-error-due-to-new-strlcpy-function-th.patch
@@ -26,2 +30,0 @@
-+
d/p/lp2038249-Exclude-zero-entries-from-do_maple_tree-return-value.patch
-+ d/p/lp2038249-Fix-irq-a-s-options-on-Linux-6.5-rc1-and-later.patch
@@ -32 +35 @@
-+ kernel 6.5 (LP: #2038249)
++ kernel 6.2 (LP: #2038249)
...
[3]
$ grep 'Origin: ' lp2038249-crash-mantic.debdiff | sed 's,.*/,,' | while read
commit; do echo -n "COMMIT $commit is in "; GIT_DIR=~/git/crash/.git git
describe --contains $commit; done
COMMIT 222176a0a6c14b6a1cdcebb8dda020ccb17b90f8 is in 8.0.3~24
COMMIT 46344aa2f92b07ded52cf9841f8db24dd7fe67d7 is in 8.0.3~21
COMMIT d17d51a92a3a1c1cce1e646c38fe52ca99406cf9 is in 8.0.4~26
COMMIT 55a43bcefa20161c7e56ed0e309e90e941f47efc is in 8.0.4~2
COMMIT 4ee56105881d7bb1da1e668ac5bb47a4e0846676 is in 8.0.4~29
COMMIT 58c1816521c2e6bece3d69256b1866c9df8d93aa is in 8.0.4~42
COMMIT 88580068b7dd96bf679c82bdc05e146968ade10c is in 8.0.4~30
COMMIT 41d4b85ea50efc733df65ec8421a74be10e47987 is in 8.0.3~31
COMMIT f182d08bab202dddf20b742fef6cc2bda0a56d6c is in 8.0.3~43
COMMIT 6d0be1316aa3666895c0a8a0d3c98c235ec03bd4 is in 8.0.4~28
COMMIT 38d35bd1423ccafd0b8be0744155ce59ef3034ff is in 8.0.4~25
COMMIT 489093c2183f4f0365d8957e7275cd88225942ce is in 8.0.3~8
COMMIT 342cf340ed0386880fe2a3115d6bef32eabb511b is in 8.0.4~41
COMMIT 0172e35083b545fa7dd640fa5de0111f8474fc14 is in 8.0.4~5
COMMIT 9efc1f68a44f6fe521e64efe4a3dc36e9ba0bbc1 is in 8.0.3~23
COMMIT 872cad2d63b3a07f65323fe80a7abb29ea276b44 is in 8.0.3~26
COMMIT 120d6e89fc14eb7f1c9a3106305c7066730f36b8 is in 8.0.3~28
COMMIT ac96e17d1de51016ee1a983e68c7e840ff55ab8d is in 8.0.3~27
COMMIT d83df2fb66cd77877d365fda32cd45c531796599 is in 8.0.3~32
COMMIT 7750e61fdb2a083f26156a5338aa2ebe26447f3f is in 8.0.4~31
COMMIT c9a732d0f6abe8c63f19fee5233544633dfd309f is in 8.0.4~11
COMMIT 141e75f3c11cc9342f11418e0bec86877424bef8 is in 8.0.3~46
COMMIT 77d8621876c1c6a3a25b91e464ba588a542485fb is in 8.0.4~36
COMMIT df1f0cba729fa0e0d8a63220769c42cc9033acc1 is in 8.0.3~44
$ grep 'Origin: ' lp2038249-crash-mantic.debdiff | sed 's,.*/,,' | while read
commit; do echo -n "COMMIT $commit is in "; GIT_DIR=~/git/crash/.git git
describe --contains $commit; done | grep -c 8.0.4
12
[4]
https://wiki.ubuntu.com/SecurityTeam/UpdatePreparation#Update_the_packaging
** Changed in: crash (Ubuntu Mantic)
Status: In Progress => Incomplete
** Changed in: crash (Ubuntu Lunar)
Status: In Progress => Incomplete
** Changed in: crash (Ubuntu Jammy)
Status: In Progress => Incomplete
** Changed in: crash (Ubuntu)
Assignee: Chengen Du (chengendu) => Mauricio Faria de Oliveira (mfo)
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to crash in Ubuntu.
https://bugs.launchpad.net/bugs/2038249
Title:
The dump file parsing issue arises from structural changes in Linux
kernel 6.2
Status in crash package in Ubuntu:
In Progress
Status in crash source package in Jammy:
Incomplete
Status in crash source package in Lunar:
Incomplete
Status in crash source package in Mantic:
Incomplete
Bug description:
[Impact]
Linux kernel 6.2 includes patches with structural changes that may render the
crash utility unable to parse the dump file.
==========
d122019bf061 mm: Split slab into its own type
401fb12c68c2 mm: Differentiate struct slab fields by sl*b implementations
07f910f9b729 mm: Remove slab from struct page
0d9b1ffefabe arm64: mm: make vabits_actual a build time constant if possible
e36ce448a08d mm/slab: use kmalloc_node() for off slab freelist_idx_t array
allocation
130d4df57390 mm/sl[au]b: rearrange struct slab fields to allow larger rcu_head
ac3b43283923 module: replace module_layout with module_memory
b69f0aeb0689 pid: Replace struct pid 1-element array with flex-array
==========
[Fix]
It is advisable to adopt commits that address the structural changes issue.
==========
In 8.0.1:
- 14f8c460473c memory: Handle struct slab changes on Linux 5.17-rc1 and later
- 5f390ed811b0 Fix for "kmem -s|-S" and "bt -F[F]" on Linux 5.17-rc1
- b89f9ccf511a Fix for "kmem -s|-S" on Linux 5.17+ with CONFIG_SLAB
In 8.0.2:
- f02c8e87fccb arm64: use TCR_EL1_T1SZ to get the correct info if
vabits_actual is missing
In 8.0.3:
- d83df2fb66cd SLUB: Fix for offset change of struct slab members on Linux
6.2-rc1
- df1f0cba729f x86_64: Fix for move of per-cpu variables into struct pcpu_hot
- 120d6e89fc14 SLAB: Fix for "kmem -s|-S" options on Linux 6.1 and later
- ac96e17d1de5 SLAB: Fix for "kmem -s|-S" options on Linux 6.2-rc1 and later
In 8.0.3++ (8.0.4 development)
- 7750e61fdb2a Support module memory layout change on Linux 6.4
- 88580068b7dd Fix failure of gathering task table on Linux 6.5-rc1 and later
- 4ee56105881d Fix compilation error due to new strlcpy function that glibc
added
==========
[Test Plan]
1. Install the required packages and then proceed to reboot the machine.
# sudo apt install crash linux-crashdump -y
# reboot
2. To check the status of kdump, use the `kdump-config show` command.
# kdump-config show
DUMP_MODE: kdump
USE_KDUMP: 1
KDUMP_COREDIR: /var/crash
crashkernel addr: 0x64000000
/var/lib/kdump/vmlinuz: symbolic link to /boot/vmlinuz-6.2.0-33-generic
kdump initrd:
/var/lib/kdump/initrd.img: symbolic link to
/var/lib/kdump/initrd.img-6.2.0-33-generic
current state: ready to kdump
kexec command:
/sbin/kexec -p --command-line="BOOT_IMAGE=/boot/vmlinuz-6.2.0-33-generic
root=UUID=3e72f5d5-870b-4b8e-9a0d-8ba920391379 ro console=tty1 console=ttyS0
reset_devices systemd.unit=kdump-tools-dump.service nr_cpus=1 irqpoll
usbcore.nousb" --initrd=/var/lib/kdump/initrd.img /var/lib/kdump/vmlinuz
3. To trigger a crash dump forcefully, execute the `echo c | sudo tee
/proc/sysrq-trigger` command.
4. Download the kernel .ddeb file, which will be used for analyzing the dump
file.
# sudo -i
# cd /var/crash
# pull-lp-ddebs linux-image-unsigned-$(uname -r)
# dpkg-deb -x linux-image-unsigned-$(uname -r)-*.ddeb dbgsym-$(uname -r)
5. Utilize the "crash" utility to parse and analyze the dump file.
crash 8.0.0
Copyright (C) 2002-2021 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011, 2020-2021 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
Copyright (C) 2015, 2021 VMware, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.
WARNING: VA_BITS: calculated: 46 vmcoreinfo: 48
GNU gdb (GDB) 10.2
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "aarch64-unknown-linux-gnu".
Type "show configuration" for configuration details.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
crash: seek error: kernel virtual address: ffffd59a92d48ae8 type: "possible"
WARNING: cannot read cpu_possible_map
crash: seek error: kernel virtual address: ffffd59a92d48b68 type: "present"
WARNING: cannot read cpu_present_map
crash: seek error: kernel virtual address: ffffd59a92d48aa8 type: "online"
WARNING: cannot read cpu_online_map
crash: seek error: kernel virtual address: ffffd59a92d48bb0 type: "active"
WARNING: cannot read cpu_active_map
crash: seek error: kernel virtual address: ffffd59a93288928 type:
"shadow_timekeeper xtime_sec"
crash: seek error: kernel virtual address: ffffd59a9317b8f0 type:
"init_uts_ns"
crash: dbgsym-6 and 202309251539/dump.202309251539 do not match!
Usage:
crash [OPTION]... NAMELIST MEMORY-IMAGE[@ADDRESS] (dumpfile form)
crash [OPTION]... [NAMELIST] (live system form)
Enter "crash -h" for details.
[Where problems could occur]
Significant structural changes have occurred between Linux kernel versions
5.15 and 6.2.
We are only incorporating patches to ensure the functionality of the "crash"
command.
However, please be aware that these patches will alter the parsing logic and
could potentially result in the "crash" utility being unable to parse the dump
file in the worst-case scenario.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/crash/+bug/2038249/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
