[Kernel-packages] [Bug 2043841] Re: kernel BUG: io_uring openat triggers audit reference count underflow
Please let me know if testing from the Canonical Kernel PPA is
sufficient or if I should test again using -proposed.
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/2043841
Title:
kernel BUG: io_uring openat triggers audit reference count underflow
Status in linux package in Ubuntu:
Fix Released
Status in linux source package in Lunar:
Fix Committed
Status in linux source package in Mantic:
Fix Released
Bug description:
I first encountered a bug in 6.2.0-1012-azure #12~22.04.1-Ubuntu that
occurs during io_uring openat audit processing. I have a kernel patch
that was accepted into the upstream kernel as well as the v6.6,
v6.5.9, and v6.1.60 releases. The bug was first introduced in the
upstream v5.16 kernel.
I do not see the change yet in:
* The Ubuntu-azure-6.2-6.2.0-1017.17_22.04.1 tag in the jammy kernel
repository.
* The Ubuntu-azure-6.5.0-1009.9 tag in the mantic kernel repository.
Can this upstream commit be cherry picked?
The upstream commit is:
03adc61edad49e1bbecfb53f7ea5d78f398fe368
The upstream patch thread is:
https://lore.kernel.org/audit/20231012215518.ga4...@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net/T/#u
The maintainer pull request thread is:
https://lore.kernel.org/lkml/20231019-kampfsport-
metapher-e5211d7be247@brauner
The pre-patch discussion thread is:
https://lore.kernel.org/io-
uring/mw2pr2101mb1033fff044a258f84aeaa584f1...@mw2pr2101mb1033.namprd21.prod.outlook.com/T/#u
The commit log message is:
commit 03adc61edad49e1bbecfb53f7ea5d78f398fe368
Author: Dan Clash
Date: Thu Oct 12 14:55:18 2023 -0700
audit,io_uring: io_uring openat triggers audit reference count
underflow
An io_uring openat operation can update an audit reference count
from multiple threads resulting in the call trace below.
A call to io_uring_submit() with a single openat op with a flag of
IOSQE_ASYNC results in the following reference count updates.
These first part of the system call performs two increments that
do not race.
do_syscall_64()
__do_sys_io_uring_enter()
io_submit_sqes()
io_openat_prep()
__io_openat_prep()
getname()
getname_flags() /* update 1 (increment) */
__audit_getname() /* update 2 (increment) */
The openat op is queued to an io_uring worker thread which starts the
opportunity for a race. The system call exit performs one decrement.
do_syscall_64()
syscall_exit_to_user_mode()
syscall_exit_to_user_mode_prepare()
__audit_syscall_exit()
audit_reset_context()
putname() /* update 3 (decrement) */
The io_uring worker thread performs one increment and two decrements.
These updates can race with the system call decrement.
io_wqe_worker()
io_worker_handle_work()
io_wq_submit_work()
io_issue_sqe()
io_openat()
io_openat2()
do_filp_open()
path_openat()
__audit_inode() /* update 4 (increment) */
putname() /* update 5 (decrement) */
__audit_uring_exit()
audit_reset_context()
putname() /* update 6 (decrement) */
The fix is to change the refcnt member of struct audit_names
from int to atomic_t.
kernel BUG at fs/namei.c:262!
Call Trace:
...
? putname+0x68/0x70
audit_reset_context.part.0.constprop.0+0xe1/0x300
__audit_uring_exit+0xda/0x1c0
io_issue_sqe+0x1f3/0x450
? lock_timer_base+0x3b/0xd0
io_wq_submit_work+0x8d/0x2b0
? __try_to_del_timer_sync+0x67/0xa0
io_worker_handle_work+0x17c/0x2b0
io_wqe_worker+0x10a/0x350
Cc: sta...@vger.kernel.org
Link:
https://lore.kernel.org/lkml/mw2pr2101mb1033fff044a258f84aeaa584f1...@mw2pr2101mb1033.namprd21.prod.outlook.com/
Fixes: 5bd2182d58e9 ("audit,io_uring,io-wq: add some basic audit support
to io_uring")
Signed-off-by: Dan Clash
Link:
https://lore.kernel.org/r/20231012215518.ga4...@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net
Reviewed-by: Jens Axboe
Signed-off-by: Christian Brauner
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2043841/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 2043841] Re: kernel BUG: io_uring openat triggers audit reference count underflow
I previously verified that the test program hangs when 6.5.0-1011-azure is installed. I have been testing with 6.5.0-1012-azure from the Canonical Kernel PPA for a while with no issues. I upgraded to 6.5.0-1013-azure just now and the test program still passes. devvm7 ~ $ uname -a Linux daclashlinux7 6.5.0-1013-azure #13~22.04.1-Ubuntu SMP Tue Feb 6 20:34:09 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux devvm7 ~ $ sudo dmesg --clear devvm7 ~ $ ./io_uring_open_close_audit_hang --directory /tmp/deleteme --count 1 i=0 i=100 i=200 ... i=9800 i=9900 devvm7 ~ $ sudo dmesg devvm7 ~ $ The test program does not hang when running with 6.5.0-1012-azure. daclash@daclashlinux4:~$ uname -a Linux daclashlinux4 6.5.0-1012-azure #12~22.04.1-Ubuntu SMP Tue Jan 16 21:24:44 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux daclash@daclashlinux4:~$ sudo dmesg --clear daclash@daclashlinux4:~$ ./io_uring_open_close_audit_hang --directory /tmp/deleteme --count 1 ... i=9900 daclash@daclashlinux4:~$ sudo dmesg daclash@daclashlinux4:~$ The test program does hang when running with 6.5.0-1011-azure. daclash@daclashlinux4:~$ uname -a Linux daclashlinux4 6.5.0-1011-azure #11~22.04.1-Ubuntu SMP Mon Jan 15 16:59:12 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux daclash@daclashlinux4:~$ sudo dmesg --clear daclash@daclashlinux4:~$ ./io_uring_open_close_audit_hang --directory /tmp/deleteme --count 1 i=0 ... i=5900 i=6000 ^C daclash@daclashlinux4:~$ sudo dmesg | grep "kernel BUG at fs/namei.c" [ 125.159601] kernel BUG at fs/namei.c:264! -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2043841 Title: kernel BUG: io_uring openat triggers audit reference count underflow Status in linux package in Ubuntu: Fix Released Status in linux source package in Lunar: Fix Committed Status in linux source package in Mantic: Fix Released Bug description: I first encountered a bug in 6.2.0-1012-azure #12~22.04.1-Ubuntu that occurs during io_uring openat audit processing. I have a kernel patch that was accepted into the upstream kernel as well as the v6.6, v6.5.9, and v6.1.60 releases. The bug was first introduced in the upstream v5.16 kernel. I do not see the change yet in: * The Ubuntu-azure-6.2-6.2.0-1017.17_22.04.1 tag in the jammy kernel repository. * The Ubuntu-azure-6.5.0-1009.9 tag in the mantic kernel repository. Can this upstream commit be cherry picked? The upstream commit is: 03adc61edad49e1bbecfb53f7ea5d78f398fe368 The upstream patch thread is: https://lore.kernel.org/audit/20231012215518.ga4...@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net/T/#u The maintainer pull request thread is: https://lore.kernel.org/lkml/20231019-kampfsport- metapher-e5211d7be247@brauner The pre-patch discussion thread is: https://lore.kernel.org/io- uring/mw2pr2101mb1033fff044a258f84aeaa584f1...@mw2pr2101mb1033.namprd21.prod.outlook.com/T/#u The commit log message is: commit 03adc61edad49e1bbecfb53f7ea5d78f398fe368 Author: Dan Clash Date: Thu Oct 12 14:55:18 2023 -0700 audit,io_uring: io_uring openat triggers audit reference count underflow An io_uring openat operation can update an audit reference count from multiple threads resulting in the call trace below. A call to io_uring_submit() with a single openat op with a flag of IOSQE_ASYNC results in the following reference count updates. These first part of the system call performs two increments that do not race. do_syscall_64() __do_sys_io_uring_enter() io_submit_sqes() io_openat_prep() __io_openat_prep() getname() getname_flags() /* update 1 (increment) */ __audit_getname() /* update 2 (increment) */ The openat op is queued to an io_uring worker thread which starts the opportunity for a race. The system call exit performs one decrement. do_syscall_64() syscall_exit_to_user_mode() syscall_exit_to_user_mode_prepare() __audit_syscall_exit() audit_reset_context() putname() /* update 3 (decrement) */ The io_uring worker thread performs one increment and two decrements. These updates can race with the system call decrement. io_wqe_worker() io_worker_handle_work() io_wq_submit_work() io_issue_sqe() io_openat() io_openat2() do_filp_open() path_openat() __audit_inode() /* update 4 (increment) */ putname() /* update 5 (decrement) */ __audit_uring_exit() audit_reset_context() putname() /* update 6 (decrement) */ The fix is
[Kernel-packages] [Bug 2043841] Re: kernel BUG: io_uring openat triggers audit reference count underflow
epository.
* The Ubuntu-azure-6.5.0-1009.9 tag in the mantic kernel repository.
Can this upstream commit be cherry picked?
The upstream commit is:
03adc61edad49e1bbecfb53f7ea5d78f398fe368
The upstream patch thread is:
https://lore.kernel.org/audit/20231012215518.ga4...@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net/T/#u
The maintainer pull request thread is:
https://lore.kernel.org/lkml/20231019-kampfsport-
metapher-e5211d7be247@brauner
The pre-patch discussion thread is:
https://lore.kernel.org/io-
uring/mw2pr2101mb1033fff044a258f84aeaa584f1...@mw2pr2101mb1033.namprd21.prod.outlook.com/T/#u
The commit log message is:
commit 03adc61edad49e1bbecfb53f7ea5d78f398fe368
Author: Dan Clash
Date: Thu Oct 12 14:55:18 2023 -0700
audit,io_uring: io_uring openat triggers audit reference count
underflow
An io_uring openat operation can update an audit reference count
from multiple threads resulting in the call trace below.
A call to io_uring_submit() with a single openat op with a flag of
IOSQE_ASYNC results in the following reference count updates.
These first part of the system call performs two increments that
do not race.
do_syscall_64()
__do_sys_io_uring_enter()
io_submit_sqes()
io_openat_prep()
__io_openat_prep()
getname()
getname_flags() /* update 1 (increment) */
__audit_getname() /* update 2 (increment) */
The openat op is queued to an io_uring worker thread which starts the
opportunity for a race. The system call exit performs one decrement.
do_syscall_64()
syscall_exit_to_user_mode()
syscall_exit_to_user_mode_prepare()
__audit_syscall_exit()
audit_reset_context()
putname() /* update 3 (decrement) */
The io_uring worker thread performs one increment and two decrements.
These updates can race with the system call decrement.
io_wqe_worker()
io_worker_handle_work()
io_wq_submit_work()
io_issue_sqe()
io_openat()
io_openat2()
do_filp_open()
path_openat()
__audit_inode() /* update 4 (increment) */
putname() /* update 5 (decrement) */
__audit_uring_exit()
audit_reset_context()
putname() /* update 6 (decrement) */
The fix is to change the refcnt member of struct audit_names
from int to atomic_t.
kernel BUG at fs/namei.c:262!
Call Trace:
...
? putname+0x68/0x70
audit_reset_context.part.0.constprop.0+0xe1/0x300
__audit_uring_exit+0xda/0x1c0
io_issue_sqe+0x1f3/0x450
? lock_timer_base+0x3b/0xd0
io_wq_submit_work+0x8d/0x2b0
? __try_to_del_timer_sync+0x67/0xa0
io_worker_handle_work+0x17c/0x2b0
io_wqe_worker+0x10a/0x350
Cc: sta...@vger.kernel.org
Link:
https://lore.kernel.org/lkml/mw2pr2101mb1033fff044a258f84aeaa584f1...@mw2pr2101mb1033.namprd21.prod.outlook.com/
Fixes: 5bd2182d58e9 ("audit,io_uring,io-wq: add some basic audit support
to io_uring")
Signed-off-by: Dan Clash
Link:
https://lore.kernel.org/r/20231012215518.ga4...@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net
Reviewed-by: Jens Axboe
Signed-off-by: Christian Brauner
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2043841/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 2043841] Re: kernel BUG: io_uring openat triggers audit reference count underflow
This bug is a in the Linux kernel, specifically in the filesystem /
io_uring / audit areas.
** Package changed: ubuntu => linux (Ubuntu)
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/2043841
Title:
kernel BUG: io_uring openat triggers audit reference count underflow
Status in linux-azure-6.2 package in Ubuntu:
New
Bug description:
I first encountered a bug in 6.2.0-1012-azure #12~22.04.1-Ubuntu that
occurs during io_uring openat audit processing. I have a kernel patch
that was accepted into the upstream kernel as well as the v6.6,
v6.5.9, and v6.1.60 releases. The bug was first introduced in the
upstream v5.16 kernel.
I do not see the change yet in:
* The Ubuntu-azure-6.2-6.2.0-1017.17_22.04.1 tag in the jammy kernel
repository.
* The Ubuntu-azure-6.5.0-1009.9 tag in the mantic kernel repository.
Can this upstream commit be cherry picked?
The upstream commit is:
03adc61edad49e1bbecfb53f7ea5d78f398fe368
The upstream patch thread is:
https://lore.kernel.org/audit/20231012215518.ga4...@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net/T/#u
The maintainer pull request thread is:
https://lore.kernel.org/lkml/20231019-kampfsport-
metapher-e5211d7be247@brauner
The pre-patch discussion thread is:
https://lore.kernel.org/io-
uring/mw2pr2101mb1033fff044a258f84aeaa584f1...@mw2pr2101mb1033.namprd21.prod.outlook.com/T/#u
The commit log message is:
commit 03adc61edad49e1bbecfb53f7ea5d78f398fe368
Author: Dan Clash
Date: Thu Oct 12 14:55:18 2023 -0700
audit,io_uring: io_uring openat triggers audit reference count
underflow
An io_uring openat operation can update an audit reference count
from multiple threads resulting in the call trace below.
A call to io_uring_submit() with a single openat op with a flag of
IOSQE_ASYNC results in the following reference count updates.
These first part of the system call performs two increments that
do not race.
do_syscall_64()
__do_sys_io_uring_enter()
io_submit_sqes()
io_openat_prep()
__io_openat_prep()
getname()
getname_flags() /* update 1 (increment) */
__audit_getname() /* update 2 (increment) */
The openat op is queued to an io_uring worker thread which starts the
opportunity for a race. The system call exit performs one decrement.
do_syscall_64()
syscall_exit_to_user_mode()
syscall_exit_to_user_mode_prepare()
__audit_syscall_exit()
audit_reset_context()
putname() /* update 3 (decrement) */
The io_uring worker thread performs one increment and two decrements.
These updates can race with the system call decrement.
io_wqe_worker()
io_worker_handle_work()
io_wq_submit_work()
io_issue_sqe()
io_openat()
io_openat2()
do_filp_open()
path_openat()
__audit_inode() /* update 4 (increment) */
putname() /* update 5 (decrement) */
__audit_uring_exit()
audit_reset_context()
putname() /* update 6 (decrement) */
The fix is to change the refcnt member of struct audit_names
from int to atomic_t.
kernel BUG at fs/namei.c:262!
Call Trace:
...
? putname+0x68/0x70
audit_reset_context.part.0.constprop.0+0xe1/0x300
__audit_uring_exit+0xda/0x1c0
io_issue_sqe+0x1f3/0x450
? lock_timer_base+0x3b/0xd0
io_wq_submit_work+0x8d/0x2b0
? __try_to_del_timer_sync+0x67/0xa0
io_worker_handle_work+0x17c/0x2b0
io_wqe_worker+0x10a/0x350
Cc: sta...@vger.kernel.org
Link:
https://lore.kernel.org/lkml/mw2pr2101mb1033fff044a258f84aeaa584f1...@mw2pr2101mb1033.namprd21.prod.outlook.com/
Fixes: 5bd2182d58e9 ("audit,io_uring,io-wq: add some basic audit support
to io_uring")
Signed-off-by: Dan Clash
Link:
https://lore.kernel.org/r/20231012215518.ga4...@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net
Reviewed-by: Jens Axboe
Signed-off-by: Christian Brauner
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux-azure-6.2/+bug/2043841/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
