vmw_prime_import_sg_table() creates a BO for a foreign dma-buf through
drm_gem_prime_import_dev(), which attaches to the dma-buf, maps the
attachment and takes a reference on it. vmw_bo_free() never undoes
this, so each foreign import leaks the attachment, its sg table and the
dma-buf reference, and the exporter's pages stay pinned after the GEM
handle, the DRM file and the importing process are gone.
Any client that can open a render node and obtain a dma-buf from
another exporter (udmabuf, V4L2, another DRM device) can pin memory
without bound. KWin does so on its own in VMware guests: it wraps
wl_shm client buffers in udmabufs and imports them, leaking gigabytes
within days.
Call drm_prime_gem_destroy() for imported objects before releasing the
GEM object, as amdgpu, radeon and nouveau do. vmw_bo_free() is TTM's
destroy callback, so TTM is done with the reservation object shared
with the dma-buf by then.
Tested on 7.1.13 and 7.2.7 with a udmabuf import reproducer and on a
Plasma desktop, where KWin's imported client buffers are now released.
The leak was found and the fix written with the help of an LLM coding
assistant.
Fixes: b32233accefff ("drm/vmwgfx: Fix prime import/export")
Cc: [email protected]
Assisted-by: Claude:claude-fable-5.1
Signed-off-by: Adam Lewis <[email protected]>
---
Notes (dropped on apply):
Security note: per Documentation/process/threat-model.rst an unprivileged
user must not be able to make the system unresponsive through unbounded
resource exhaustion. On a default systemd desktop every logged-in user
has a render node (0666) and /dev/udmabuf (uaccess ACL), which is all
this needs, and the pinned memory survives the triggering process. The
bug was found with AI assistance, so per security-bugs.rst it is treated
as public and the reproducer is not attached (available privately).
A CVE request will follow once the fix is in a released kernel.
Environment: VMware Fusion on an Apple M3 Max host, Fedora 44 aarch64
guest, vmwgfx 2.21.0, no swap. Found on 7.1.12/7.1.13 (Mesa 26.1.8,
KWin 6.7.4), confirmed on 7.2.7 (Mesa 26.2.3, KWin 6.7.5). The import
and destructor code is unchanged from v7.1 through v7.3-rc5 and current
drm-misc-fixes; b32233accefff went to stable # v6.6+. Not the same
issue as f739416dc555 ("drm/vmwgfx: drop dma_buf reference on
foreign-fd prime import"), which covers ttm_prime_fd_to_handle()'s
error path and was already in the kernels tested.
How it shows up: KWin wraps every wl_shm client buffer in a udmabuf and
imports it into vmwgfx, for scanout and, when it renders with llvmpipe
because svga reports "No 3D enabled", as a texture. Every import
leaked, so every client buffer stayed pinned: ~6 GB after a few days on
7.1. debugfs bufinfo then listed 961 live udmabufs, 954 of them still
attached to vmwgfx, while kwin_wayland held 35 dma-buf fds. On an
unpatched 7.2.7 boot, nr_foll_pin_released did not move at all in four
hours of desktop use, while MemAvailable fell to 2.8 of 24.5 GB.
Testing:
- reproducer (udmabuf -> PRIME_FD_TO_HANDLE -> GEM_CLOSE -> close the
dma-buf -> close the DRM fd): unpatched, the dma-buf outlives the
process with refcount 1 and vmwgfx still attached; patched (7.1.13
and 7.2.7), the pins are released as soon as the dma-buf fd is
closed;
- imported framebuffer: ADDFB2 + SETPLANE on/off + CLOSEFB through the
screen-target path on 7.1.13, and ADDFB2 + CLOSEFB on 7.2.7, both
release cleanly;
- Plasma desktop on 7.2.7 with the patch, 30 terminal windows opened
and closed: KWin created 298 udmabufs and 296 were freed (the other
two were still in use), drm_prime_gem_destroy() ran for 85 imported
objects, and the pin counters, slab usage and dma-buf census went
back to their baseline;
- builds without warnings with W=1 on drm-misc-fixes (e78a9fb).
Two related leaks on the same desktop, to be reported separately:
- Mesa's kms-dri-sw winsys releases GEM handles with MODE_DESTROY_DUMB,
which is EACCES on render nodes, so a compositor rendering with
llvmpipe on a render node leaks the handle of every imported buffer
(KWin up to 6.7.4; 6.7.5 falls back to the KMS node);
- vmw_buffer_prime_to_surface_base() creates a GEM handle the caller
never sees and WARNs for every foreign dma-buf.
With this fix those leaks last until the owning process exits; without
it they pin memory permanently too.
drivers/gpu/drm/vmwgfx/vmwgfx_bo.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_bo.c
b/drivers/gpu/drm/vmwgfx/vmwgfx_bo.c
index 9c7a73c..f226b32 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_bo.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_bo.c
@@ -31,6 +31,7 @@
#include "vmwgfx_resource_priv.h"
#include <drm/ttm/ttm_placement.h>
+#include <drm/drm_prime.h>
/**
* vmw_bo_free - vmw_bo destructor
@@ -69,6 +70,8 @@ static void vmw_bo_free(struct ttm_buffer_object *bo)
vmw_surface_unreference(&vbo->dumb_surface);
}
WARN_ON(!RB_EMPTY_ROOT(&vbo->res_tree));
+ if (drm_gem_is_imported(&vbo->tbo.base))
+ drm_prime_gem_destroy(&vbo->tbo.base, vbo->tbo.sg);
drm_gem_object_release(&vbo->tbo.base);
WARN_ON(vbo->dirty);
kfree(vbo);
base-commit: e78a9fb7a40c55ec2a70ae39903dde4f898af2ef
--
2.55.0
