Profiling NFSD under an iozone workload showed that hardened
usercopy checks consume roughly 1.3% of CPU in the TCP receive
path. The runtime check in check_object_size() validates that
copy buffers reside in expected slab regions, which is
meaningful when data crosses the user/kernel boundary but adds
no value when both source and destination are kernel addresses.
The fix splits check_copy_size() into two variants: the
existing full check, and a new __compiletime_check_copy_size()
that retains the compile-time object size assertion and the
runtime overflow check but omits check_object_size(). A
user_backed_iter() test at each call site selects between
them, so user-backed iterators continue to receive the full
validation.
Patch 1 applies this to copy_to_iter(). Patch 2 applies the
same change to copy_from_iter(). copy_from_iter_nocache() is
left unchanged because all current callers pass user-space
addresses; the bypass there is deferred until that changes.
---
Changes since v1:
- Updated commit message for clarity and completeness
- Rename the check_copy_size() function (Kees)
- Added __compiletime_check_copy_size() stub to
tools/virtio/linux/ucopysize.h
- Add second patch to convert copy_from_iter()
---
Chuck Lever (2):
iov: Bypass usercopy hardening for copy_to_iter()
iov: Bypass usercopy hardening for copy_from_iter()
include/linux/ucopysize.h | 16 +++++++++++++++-
include/linux/uio.h | 18 ++++++++++++++----
tools/virtio/linux/ucopysize.h | 6 ++++++
3 files changed, 35 insertions(+), 5 deletions(-)
---
base-commit: 7aaa8047eafd0bd628065b15757d9b48c5f9c07d
change-id: 20260326-bypass-user-copy-3e73161cc90b
Best regards,
--
Chuck Lever
