Patch request submitted for focal:
https://lists.ubuntu.com/archives/kernel-team/2021-October/thread.html#124589
changing status to 'In Progress' for focal.
** Changed in: linux (Ubuntu Focal)
Status: Incomplete => In Progress
** Changed in: ubuntu-z-systems
Status: Incomplete => In Progress
** Changed in: linux (Ubuntu Focal)
Assignee: Frank Heimes (fheimes) => Canonical Kernel Team
(canonical-kernel-team)
** Description changed:
+ SRU Justification:
+ ==================
+
+ [Impact]
+
+ * Problems occur in IBM z/VM's IUCV (Inter User Communication Vehicle)
+ environments and its communication.
+
+ * Errors like "usercopy: Kernel memory overwrite attempt detected to
+ SLUB object 'dma-kmalloc-1 k' (offset 0, size 11)!" pop up and cause
+ failures.
+
+ * This is because IUCV uses kmalloc() with __GFP_DMA because of memory
+ address restrictions.
+
+ * The solution is to mark dma-kmalloc caches as usercopy caches.
+
+ [Fix]
+
+ * 49f2d2419d60a103752e5fbaf158cf8d07c0d884 49f2d2419d60 "usercopy: mark
+ dma-kmalloc caches as usercopy caches"
+
+ * Due to changes in the context of the upstream patch,
+ a cherry-pick was not possible and the following backport was created:
+
https://bugs.launchpad.net/bugs/1913442/+attachment/5457885/+files/commit_49f2d2419d60_backport.patch
+
+ [Test Case]
+
+ * Setup Ubuntu Server 20.04 on s390x system as IBM z/VM guest aka
+ virtual machine.
+
+ * Setup IUCV on z/VM: Setting up the (IUCV TCPIP) service machine:
+
https://www.ibm.com/support/knowledgecenter/linuxonibm/com.ibm.linux.z.ljdd/ljdd_t_iucv_tcpservice.html
+
+ * Set up a Linux IUCV instance:
+
https://www.ibm.com/support/knowledgecenter/linuxonibm/com.ibm.linux.z.ljdd/ljdd_t_iucv_scen1_guest.html
+
+ * Set up an IUCV direct:
+
https://www.ibm.com/support/knowledgecenter/linuxonibm/com.ibm.linux.z.ljdd/ljdd_c_iucv_connect.html
+
+ * Make use of IUCV, for example using ssh on a direct connection.
+
+ * Verify if the connections is stable and watch out for messages
+ starting with "usercopy".
+
+ [Regression Potential]
+
+ * Problems could occur in case the create_kmalloc_cache call is done wrong,
+ for example with wrong index, wrong size or just wrong comma separations.
+
+ * Wrong size or index will probably lead to similar instability problems
+ that exist today.
+
+ * Problems in the syntax (commas etc.) will be detected at compile time.
+
+ * But it's just a single line modification in function
+ create_kmalloc_caches of /mm/slab_common.c,
+
+ * so the change is very limited and quite traceable.
+
+ * And it was in depth discussed here:
+
https://lore.kernel.org/kernel-hardening/1515636190-24061-2-git-send-email-keesc...@chromium.org/
+
+ * a reviewed by a lot of kernel engineers (see provenance)
+
+ * and it was already upstream accepted with kernel 5.8.
+
+ [Other]
+
+ * Since the commit is upstream accepted with 5.8, so it's already in
+ impish and hirsute (and groovy).
+
+ * Hence this kernel SRU submission is for Focal only and covers only the
above single but common code commit/patch.
+ __________
+
When I deployed a Ubuntu20.04 instance with kernel version of
5.4.0-58-generic under z/VM, I saw below messages from kernel and the
iucvserv program malfunctioned. Hence it caused some devices like
network device configuration failure and deployment failure.
-
Dec 14 22:02:26 ub2004img iucvserv: Receive OPNCLD4 0.0.0.1 pwd sent from
IUCV client.
Dec 14 22:02:26 ub2004img iucvserv: /etc/iucv_authorized_userid exists, check
authorization.
Dec 14 22:02:26 ub2004img iucvserv: senduserid=OPNCLD4, authuserid=OPNCLD4,
len=7
Dec 14 22:02:26 ub2004img iucvserv: Current version is 0.0.0.1, upgraded
version is 0.0.0.1
Dec 14 22:02:26 ub2004img iucvserv: Will execute the linux command pwd 2>&1;
echo iucvcmdrc=$? sent from IUCV client.
Dec 14 22:02:26 ub2004img iucvserv: result length=14, send message
length=14,#012 /#012iucvcmdrc=0
Dec 14 22:02:26 ub2004img kernel: [63084.184649] ------------[ cut here
]------------
Dec 14 22:02:26 ub2004img kernel: [63084.184654] Bad or missing usercopy
whitelist? Kernel memory exposure attempt detected from SLUB object
'dma-kmalloc-1k' (offset 0, size 20)!
Dec 14 22:02:26 ub2004img kernel: [63084.184680] WARNING: CPU: 1 PID: 697 at
mm/usercopy.c:75 usercopy_warn+0xa0/0xd0
Dec 14 22:02:26 ub2004img kernel: [63084.184681] Modules linked in: tcp_diag
udp_diag raw_diag inet_diag unix_diag xt_CT iptable_raw ipt_REJECT
nf_reject_ipv4 xt_tcpudp xt_conntrack nf_conntrack nf_defr
ag_ipv6 nf_defrag_ipv4 iptable_filter bpfilter af_iucv nls_utf8 isofs
dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua vmur vfio_ccw vfio_mdev mdev
s390_trng vfio_iommu_type1 vfio sch_fq_codel drm drm
_panel_orientation_quirks i2c_core ip_tables x_tables btrfs zstd_compress
zlib_deflate raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor
async_tx xor raid6_pq libcrc32c raid1 raid0 linear
- pkey zcrypt crc32_vx_s390 ghash_s390 prng aes_s390 des_s390 libdes
sha3_512_s390 sha3_256_s390 sha512_s390 sha256_s390 sha1_s390 sha_common
dasd_fba_mod dasd_mod qeth_l2 qeth qdio ccwgroup
+ pkey zcrypt crc32_vx_s390 ghash_s390 prng aes_s390 des_s390 libdes
sha3_512_s390 sha3_256_s390 sha512_s390 sha256_s390 sha1_s390 sha_common
dasd_fba_mod dasd_mod qeth_l2 qeth qdio ccwgroup
Dec 14 22:02:26 ub2004img kernel: [63084.184718] CPU: 1 PID: 697 Comm:
iucvserv Not tainted 5.4.0-58-generic #64-Ubuntu
Dec 14 22:02:26 ub2004img kernel: [63084.184718] Hardware name: IBM 8561 LT1
400 (z/VM 7.1.0)
Dec 14 22:02:26 ub2004img kernel: [63084.184719] Krnl PSW : 0704c00180000000
00000000b3c37a60 (usercopy_warn+0xa0/0xd0)
Dec 14 22:02:26 ub2004img kernel: [63084.184721] R:0 T:1 IO:1 EX:1
Key:0 M:1 W:0 P:0 AS:3 CC:0 PM:0 RI:0 EA:3
Dec 14 22:02:26 ub2004img kernel: [63084.184722] Krnl GPRS: 0000000000000004
0000000000000006 0000000000000081 0000000000000007
Dec 14 22:02:26 ub2004img kernel: [63084.184722] 0000000000000007
00000000f2ecb400 00000000b43fdc6a 000003e000000014
Dec 14 22:02:26 ub2004img kernel: [63084.184723] 0000000000000000
0000000000000014 0000000000000000 00000000b43f01f0
Dec 14 22:02:26 ub2004img kernel: [63084.184723] 00000000aae13300
00000000e9332a00 00000000b3c37a5c 000003e000987a10
Dec 14 22:02:26 ub2004img kernel: [63084.184728] Krnl Code: 00000000b3c37a50:
c020003e310f larl %r2,00000000b43fdc6e
Dec 14 22:02:26 ub2004img kernel: [63084.184728] 00000000b3c37a56:
c0e5ffedbe85 brasl %r14,00000000b39ef760
Dec 14 22:02:26 ub2004img kernel: [63084.184728] #00000000b3c37a5c:
a7f40001 brc 15,00000000b3c37a5e
Dec 14 22:02:26 ub2004img kernel: [63084.184728] >00000000b3c37a60:
eb6ff0c00004 lmg %r6,%r15,192(%r15)
Dec 14 22:02:26 ub2004img kernel: [63084.184728] 00000000b3c37a66:
07fe bcr 15,%r14
Dec 14 22:02:26 ub2004img kernel: [63084.184728] 00000000b3c37a68:
47000700 bc 0,1792
Dec 14 22:02:26 ub2004img kernel: [63084.184728] 00000000b3c37a6c:
c020003e30fa larl %r2,00000000b43fdc60
Dec 14 22:02:26 ub2004img kernel: [63084.184728] 00000000b3c37a72:
a7f4ffd4 brc 15,00000000b3c37a1a
Dec 14 22:02:26 ub2004img kernel: [63084.184735] Call Trace:
Dec 14 22:02:26 ub2004img kernel: [63084.184736] ([<00000000b3c37a5c>]
usercopy_warn+0x9c/0xd0)
Dec 14 22:02:26 ub2004img kernel: [63084.184740] [<00000000b3c0fcc8>]
__check_heap_object+0xd8/0x150
Dec 14 22:02:26 ub2004img kernel: [63084.184741] [<00000000b3c37bc4>]
__check_object_size+0x134/0x200
Dec 14 22:02:26 ub2004img kernel: [63084.184744] [<00000000b4080a7e>]
simple_copy_to_iter+0x3e/0x70
Dec 14 22:02:26 ub2004img kernel: [63084.184745] [<00000000b407fe02>]
__skb_datagram_iter+0x72/0x280
Dec 14 22:02:26 ub2004img kernel: [63084.184745] [<00000000b40800be>]
skb_copy_datagram_iter+0x5e/0xe0
Dec 14 22:02:26 ub2004img kernel: [63084.184747] [<000003ff805014ea>]
iucv_sock_recvmsg+0xaa/0x460 [af_iucv]
Dec 14 22:02:26 ub2004img kernel: [63084.184749] [<00000000b406ce36>]
__sys_recvfrom+0xb6/0x140
Dec 14 22:02:26 ub2004img kernel: [63084.184750] [<00000000b406e042>]
__s390x_sys_socketcall+0x222/0x350
Dec 14 22:02:26 ub2004img kernel: [63084.184753] [<00000000b4250ba2>]
system_call+0x2a6/0x2c8
Dec 14 22:02:26 ub2004img kernel: [63084.184753] Last Breaking-Event-Address:
Dec 14 22:02:26 ub2004img kernel: [63084.184754] [<00000000b3c37a5c>]
usercopy_warn+0x9c/0xd0
Dec 14 22:02:26 ub2004img kernel: [63084.184754] ---[ end trace
b0232fe5536a773d ]---
Dec 14 22:02:26 ub2004img iucvserv: Receive OPNCLD4 0.0.0.1 ls /etc/*-release
sent from IUCV client.
Dec 14 22:02:26 ub2004img iucvserv: /etc/iucv_authorized_userid exists, check
authorization.
Dec 14 22:02:26 ub2004img iucvserv: senduserid=OPNCLD4, authuserid=OPNCLD4,
len=7
Dec 14 22:02:26 ub2004img iucvserv: Current version is 0.0.0.1, upgraded
version is 0.0.0.1
Dec 14 22:02:26 ub2004img iucvserv: Will execute the linux command ls
/etc/*-release 2>&1; echo iucvcmdrc=$? sent from IUCV client.
Dec 14 22:02:26 ub2004img iucvserv: result length=45, send message
length=45,#012 /etc/lsb-release#012/etc/os-release#012iucvcmdrc=0
Dec 14 22:02:26 ub2004img iucvserv: Receive OPNCLD4 0.0.0.1 cat
/etc/os-release sent from IUCV client.
Dec 14 22:02:26 ub2004img iucvserv: /etc/iucv_authorized_userid exists, check
authorization.
Dec 14 22:02:26 ub2004img iucvserv: senduserid=OPNCLD4, authuserid=OPNCLD4,
len=7
Dec 14 22:02:26 ub2004img iucvserv: Current version is 0.0.0.1, upgraded
version is 0.0.0.1
-
- But I didn't see such problem with kernel version 5.4.0-40-generic #44-Ubuntu
when I did the same operation.
+ But I didn't see such problem with kernel version 5.4.0-40-generic
+ #44-Ubuntu when I did the same operation.
** Information type changed from Private to Public
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1913442
Title:
[Ubuntu 20.04] Problem leading IUCV service down (on s390x)
Status in Ubuntu on IBM z Systems:
In Progress
Status in linux package in Ubuntu:
Fix Released
Status in linux source package in Focal:
In Progress
Status in linux source package in Hirsute:
Fix Released
Status in linux source package in Impish:
Fix Released
Bug description:
SRU Justification:
==================
[Impact]
* Problems occur in IBM z/VM's IUCV (Inter User Communication Vehicle)
environments and its communication.
* Errors like "usercopy: Kernel memory overwrite attempt detected to
SLUB object 'dma-kmalloc-1 k' (offset 0, size 11)!" pop up and cause
failures.
* This is because IUCV uses kmalloc() with __GFP_DMA because of memory
address restrictions.
* The solution is to mark dma-kmalloc caches as usercopy caches.
[Fix]
* 49f2d2419d60a103752e5fbaf158cf8d07c0d884 49f2d2419d60 "usercopy:
mark dma-kmalloc caches as usercopy caches"
* Due to changes in the context of the upstream patch,
a cherry-pick was not possible and the following backport was created:
https://bugs.launchpad.net/bugs/1913442/+attachment/5457885/+files/commit_49f2d2419d60_backport.patch
[Test Case]
* Setup Ubuntu Server 20.04 on s390x system as IBM z/VM guest aka
virtual machine.
* Setup IUCV on z/VM: Setting up the (IUCV TCPIP) service machine:
https://www.ibm.com/support/knowledgecenter/linuxonibm/com.ibm.linux.z.ljdd/ljdd_t_iucv_tcpservice.html
* Set up a Linux IUCV instance:
https://www.ibm.com/support/knowledgecenter/linuxonibm/com.ibm.linux.z.ljdd/ljdd_t_iucv_scen1_guest.html
* Set up an IUCV direct:
https://www.ibm.com/support/knowledgecenter/linuxonibm/com.ibm.linux.z.ljdd/ljdd_c_iucv_connect.html
* Make use of IUCV, for example using ssh on a direct connection.
* Verify if the connections is stable and watch out for messages
starting with "usercopy".
[Regression Potential]
* Problems could occur in case the create_kmalloc_cache call is done wrong,
for example with wrong index, wrong size or just wrong comma separations.
* Wrong size or index will probably lead to similar instability
problems that exist today.
* Problems in the syntax (commas etc.) will be detected at compile
time.
* But it's just a single line modification in function
create_kmalloc_caches of /mm/slab_common.c,
* so the change is very limited and quite traceable.
* And it was in depth discussed here:
https://lore.kernel.org/kernel-hardening/1515636190-24061-2-git-send-email-keesc...@chromium.org/
* a reviewed by a lot of kernel engineers (see provenance)
* and it was already upstream accepted with kernel 5.8.
[Other]
* Since the commit is upstream accepted with 5.8, so it's already in
impish and hirsute (and groovy).
* Hence this kernel SRU submission is for Focal only and covers only the
above single but common code commit/patch.
__________
When I deployed a Ubuntu20.04 instance with kernel version of
5.4.0-58-generic under z/VM, I saw below messages from kernel and the
iucvserv program malfunctioned. Hence it caused some devices like
network device configuration failure and deployment failure.
Dec 14 22:02:26 ub2004img iucvserv: Receive OPNCLD4 0.0.0.1 pwd sent from
IUCV client.
Dec 14 22:02:26 ub2004img iucvserv: /etc/iucv_authorized_userid exists, check
authorization.
Dec 14 22:02:26 ub2004img iucvserv: senduserid=OPNCLD4, authuserid=OPNCLD4,
len=7
Dec 14 22:02:26 ub2004img iucvserv: Current version is 0.0.0.1, upgraded
version is 0.0.0.1
Dec 14 22:02:26 ub2004img iucvserv: Will execute the linux command pwd 2>&1;
echo iucvcmdrc=$? sent from IUCV client.
Dec 14 22:02:26 ub2004img iucvserv: result length=14, send message
length=14,#012 /#012iucvcmdrc=0
Dec 14 22:02:26 ub2004img kernel: [63084.184649] ------------[ cut here
]------------
Dec 14 22:02:26 ub2004img kernel: [63084.184654] Bad or missing usercopy
whitelist? Kernel memory exposure attempt detected from SLUB object
'dma-kmalloc-1k' (offset 0, size 20)!
Dec 14 22:02:26 ub2004img kernel: [63084.184680] WARNING: CPU: 1 PID: 697 at
mm/usercopy.c:75 usercopy_warn+0xa0/0xd0
Dec 14 22:02:26 ub2004img kernel: [63084.184681] Modules linked in: tcp_diag
udp_diag raw_diag inet_diag unix_diag xt_CT iptable_raw ipt_REJECT
nf_reject_ipv4 xt_tcpudp xt_conntrack nf_conntrack nf_defr
ag_ipv6 nf_defrag_ipv4 iptable_filter bpfilter af_iucv nls_utf8 isofs
dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua vmur vfio_ccw vfio_mdev mdev
s390_trng vfio_iommu_type1 vfio sch_fq_codel drm drm
_panel_orientation_quirks i2c_core ip_tables x_tables btrfs zstd_compress
zlib_deflate raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor
async_tx xor raid6_pq libcrc32c raid1 raid0 linear
pkey zcrypt crc32_vx_s390 ghash_s390 prng aes_s390 des_s390 libdes
sha3_512_s390 sha3_256_s390 sha512_s390 sha256_s390 sha1_s390 sha_common
dasd_fba_mod dasd_mod qeth_l2 qeth qdio ccwgroup
Dec 14 22:02:26 ub2004img kernel: [63084.184718] CPU: 1 PID: 697 Comm:
iucvserv Not tainted 5.4.0-58-generic #64-Ubuntu
Dec 14 22:02:26 ub2004img kernel: [63084.184718] Hardware name: IBM 8561 LT1
400 (z/VM 7.1.0)
Dec 14 22:02:26 ub2004img kernel: [63084.184719] Krnl PSW : 0704c00180000000
00000000b3c37a60 (usercopy_warn+0xa0/0xd0)
Dec 14 22:02:26 ub2004img kernel: [63084.184721] R:0 T:1 IO:1 EX:1
Key:0 M:1 W:0 P:0 AS:3 CC:0 PM:0 RI:0 EA:3
Dec 14 22:02:26 ub2004img kernel: [63084.184722] Krnl GPRS: 0000000000000004
0000000000000006 0000000000000081 0000000000000007
Dec 14 22:02:26 ub2004img kernel: [63084.184722] 0000000000000007
00000000f2ecb400 00000000b43fdc6a 000003e000000014
Dec 14 22:02:26 ub2004img kernel: [63084.184723] 0000000000000000
0000000000000014 0000000000000000 00000000b43f01f0
Dec 14 22:02:26 ub2004img kernel: [63084.184723] 00000000aae13300
00000000e9332a00 00000000b3c37a5c 000003e000987a10
Dec 14 22:02:26 ub2004img kernel: [63084.184728] Krnl Code: 00000000b3c37a50:
c020003e310f larl %r2,00000000b43fdc6e
Dec 14 22:02:26 ub2004img kernel: [63084.184728] 00000000b3c37a56:
c0e5ffedbe85 brasl %r14,00000000b39ef760
Dec 14 22:02:26 ub2004img kernel: [63084.184728] #00000000b3c37a5c:
a7f40001 brc 15,00000000b3c37a5e
Dec 14 22:02:26 ub2004img kernel: [63084.184728] >00000000b3c37a60:
eb6ff0c00004 lmg %r6,%r15,192(%r15)
Dec 14 22:02:26 ub2004img kernel: [63084.184728] 00000000b3c37a66:
07fe bcr 15,%r14
Dec 14 22:02:26 ub2004img kernel: [63084.184728] 00000000b3c37a68:
47000700 bc 0,1792
Dec 14 22:02:26 ub2004img kernel: [63084.184728] 00000000b3c37a6c:
c020003e30fa larl %r2,00000000b43fdc60
Dec 14 22:02:26 ub2004img kernel: [63084.184728] 00000000b3c37a72:
a7f4ffd4 brc 15,00000000b3c37a1a
Dec 14 22:02:26 ub2004img kernel: [63084.184735] Call Trace:
Dec 14 22:02:26 ub2004img kernel: [63084.184736] ([<00000000b3c37a5c>]
usercopy_warn+0x9c/0xd0)
Dec 14 22:02:26 ub2004img kernel: [63084.184740] [<00000000b3c0fcc8>]
__check_heap_object+0xd8/0x150
Dec 14 22:02:26 ub2004img kernel: [63084.184741] [<00000000b3c37bc4>]
__check_object_size+0x134/0x200
Dec 14 22:02:26 ub2004img kernel: [63084.184744] [<00000000b4080a7e>]
simple_copy_to_iter+0x3e/0x70
Dec 14 22:02:26 ub2004img kernel: [63084.184745] [<00000000b407fe02>]
__skb_datagram_iter+0x72/0x280
Dec 14 22:02:26 ub2004img kernel: [63084.184745] [<00000000b40800be>]
skb_copy_datagram_iter+0x5e/0xe0
Dec 14 22:02:26 ub2004img kernel: [63084.184747] [<000003ff805014ea>]
iucv_sock_recvmsg+0xaa/0x460 [af_iucv]
Dec 14 22:02:26 ub2004img kernel: [63084.184749] [<00000000b406ce36>]
__sys_recvfrom+0xb6/0x140
Dec 14 22:02:26 ub2004img kernel: [63084.184750] [<00000000b406e042>]
__s390x_sys_socketcall+0x222/0x350
Dec 14 22:02:26 ub2004img kernel: [63084.184753] [<00000000b4250ba2>]
system_call+0x2a6/0x2c8
Dec 14 22:02:26 ub2004img kernel: [63084.184753] Last Breaking-Event-Address:
Dec 14 22:02:26 ub2004img kernel: [63084.184754] [<00000000b3c37a5c>]
usercopy_warn+0x9c/0xd0
Dec 14 22:02:26 ub2004img kernel: [63084.184754] ---[ end trace
b0232fe5536a773d ]---
Dec 14 22:02:26 ub2004img iucvserv: Receive OPNCLD4 0.0.0.1 ls /etc/*-release
sent from IUCV client.
Dec 14 22:02:26 ub2004img iucvserv: /etc/iucv_authorized_userid exists, check
authorization.
Dec 14 22:02:26 ub2004img iucvserv: senduserid=OPNCLD4, authuserid=OPNCLD4,
len=7
Dec 14 22:02:26 ub2004img iucvserv: Current version is 0.0.0.1, upgraded
version is 0.0.0.1
Dec 14 22:02:26 ub2004img iucvserv: Will execute the linux command ls
/etc/*-release 2>&1; echo iucvcmdrc=$? sent from IUCV client.
Dec 14 22:02:26 ub2004img iucvserv: result length=45, send message
length=45,#012 /etc/lsb-release#012/etc/os-release#012iucvcmdrc=0
Dec 14 22:02:26 ub2004img iucvserv: Receive OPNCLD4 0.0.0.1 cat
/etc/os-release sent from IUCV client.
Dec 14 22:02:26 ub2004img iucvserv: /etc/iucv_authorized_userid exists, check
authorization.
Dec 14 22:02:26 ub2004img iucvserv: senduserid=OPNCLD4, authuserid=OPNCLD4,
len=7
Dec 14 22:02:26 ub2004img iucvserv: Current version is 0.0.0.1, upgraded
version is 0.0.0.1
But I didn't see such problem with kernel version 5.4.0-40-generic
#44-Ubuntu when I did the same operation.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-z-systems/+bug/1913442/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
