Re: [Intel-gfx] [PATCH v15 00/23] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers
On 8/28/23 18:24, Helen Mae Koike Fornazier wrote: > On Monday, August 28, 2023 11:37 -03, "Helen Mae Koike Fornazier" > wrote: > >> On Sunday, August 27, 2023 14:54 -03, Dmitry Osipenko >> wrote: >> >>> This series: >>> >>> 1. Adds common drm-shmem memory shrinker >>> 2. Enables shrinker for VirtIO-GPU driver >>> 3. Switches Panfrost driver to the common shrinker >> >> Hi Dmitry, >> >> Would you mind testing with drm-ci? We virt-io tests there and it would be >> really great to get your feedback of it. >> >> https://cgit.freedesktop.org/drm/drm/log/?h=topic/drm-ci > > sorry, I forgot that you also need this patchset: > https://lists.freedesktop.org/archives/dri-devel/2023-August/420063.html > to enable virtio_gpu test job. > > Thanks again. > Helen > >> >> You need to merge your changes with the above tree. >> To configure it, you just need to have a tree on gitlab.freedesktop.org, >> go to the settings and change the CI/CD configuration file from >> .gitlab-ci.yml >> to drivers/gpu/drm/ci/gitlab-ci.yml, and you can start a pipeline >> on your branch. >> >> at the time of this writting, gitlab.freedesktop.org is under maintenance, >> but it should be back soon. Thanks, Helen. I'll give it a try for the next version -- Best regards, Dmitry
Re: [Intel-gfx] [PATCH v15 00/23] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers
On Monday, August 28, 2023 11:37 -03, "Helen Mae Koike Fornazier" wrote: > On Sunday, August 27, 2023 14:54 -03, Dmitry Osipenko > wrote: > > > This series: > > > > 1. Adds common drm-shmem memory shrinker > > 2. Enables shrinker for VirtIO-GPU driver > > 3. Switches Panfrost driver to the common shrinker > > Hi Dmitry, > > Would you mind testing with drm-ci? We virt-io tests there and it would be > really great to get your feedback of it. > > https://cgit.freedesktop.org/drm/drm/log/?h=topic/drm-ci sorry, I forgot that you also need this patchset: https://lists.freedesktop.org/archives/dri-devel/2023-August/420063.html to enable virtio_gpu test job. Thanks again. Helen > > You need to merge your changes with the above tree. > To configure it, you just need to have a tree on gitlab.freedesktop.org, > go to the settings and change the CI/CD configuration file from .gitlab-ci.yml > to drivers/gpu/drm/ci/gitlab-ci.yml, and you can start a pipeline > on your branch. > > at the time of this writting, gitlab.freedesktop.org is under maintenance, > but it should be back soon. > > Thank you! > Helen > > > > > Changelog: > > > > v15:- Moved drm-shmem reference counters to use kref that allows to > > optimize unlocked functions, like was suggested by Boris Brezillon. > > > > - Changed drm/gem/shmem function names to use _locked postfix and > > dropped the _unlocked, making the naming scheme consistent across > > DRM code, like was suggested by Boris Brezillon. > > > > - Added patch that fixes UAF in drm-shmem for drivers that import > > dma-buf and then release buffer in the import error code path. > > > > - Added patch that makes drm-shmem use new flag for SGT's get_pages() > > refcounting, preventing unbalanced refcounting when GEM is freed. > > > > - Fixed guest blob pinning in virtio-gpu driver that was missed > > previously in the shrinker patch. > > > > - Moved VC4 and virtio-gpu drivers to use drm_gem_put() in > > GEM-creation error code paths, which is now required by drm-shmem > > and was missed in a previous patch versions. > > > > - Virtio-GPU now attaches shmem pages to host on first use and not > > when BO is created. In older patch versions there was a potential > > race condition in the BO creation code path where both > > get_sgt()+object_attach() should've been made under same resv lock, > > otherwise pages could be evicted before attachment is invoked. > > > > - Virtio-GPU and drm-shmem shrinker patches are split into smaller > > ones. > > > > v14:- All the prerequisite reservation locking patches landed upstream, > > previously were a part of this series in v13 and older. > > > > > > https://lore.kernel.org/dri-devel/20230529223935.2672495-1-dmitry.osipe...@collabora.com/ > > > > - Added patches to improve locked/unlocked function names, like was > > suggested by Boris Brezillon for v13. > > > > - Made all exported drm-shmem symbols GPL, like was previously > > discussed with Thomas Zimmermann on this series. > > > > - Improved virtio-gpu shrinker patch. Now it won't detach purged BO > > when userspace closes GEM. Crosvm (and not qemu) checks res_id on > > CMD_CTX_DETACH_RESOURCE and prints noisy error message if ID is > > invalid, which wasn't noticed before. > > > > v13:- Updated virtio-gpu shrinker patch to use drm_gem_shmem_object_pin() > > directly instead of drm_gem_pin() and dropped patch that exported > > drm_gem_pin() functions, like was requested by Thomas Zimmermann in > > v12. > > > > v12:- Fixed the "no previous prototype for function" warning reported by > > kernel build bot for v11. > > > > - Fixed the missing reservation lock reported by Intel CI for VGEM > > driver. Other drivers using drm-shmem were affected similarly to > > VGEM. The problem was in the dma-buf attachment code path that led > > to drm-shmem pinning function which assumed the held reservation lock > > by drm_gem_pin(). In the past that code path was causing trouble for > > i915 driver and we've changed the locking scheme for the attachment > > code path in the dma-buf core to let exporters to handle the locking > > themselves. After a closer investigation, I realized that my > > assumption > > about testing of dma-buf export code path using Panfrost driver was > > incorrect. Now I created additional local test to exrecise the > > Panfrost > > export path. I also reproduced the issue reported by the Intel CI for > > v10. It's all fixed now by making the drm_gem_shmem_pin() to take the > > resv lock by itself. > > > > - Patches are based on top of drm-tip, CC'd intel-gfx CI for testing. > > > > v11:- Rebased on a recent linux-next. Added new patch as a result: > > > > drm/shmem-helper: Export
Re: [Intel-gfx] [PATCH v15 00/23] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers
On Sunday, August 27, 2023 14:54 -03, Dmitry Osipenko wrote: > This series: > > 1. Adds common drm-shmem memory shrinker > 2. Enables shrinker for VirtIO-GPU driver > 3. Switches Panfrost driver to the common shrinker Hi Dmitry, Would you mind testing with drm-ci? We virt-io tests there and it would be really great to get your feedback of it. https://cgit.freedesktop.org/drm/drm/log/?h=topic/drm-ci You need to merge your changes with the above tree. To configure it, you just need to have a tree on gitlab.freedesktop.org, go to the settings and change the CI/CD configuration file from .gitlab-ci.yml to drivers/gpu/drm/ci/gitlab-ci.yml, and you can start a pipeline on your branch. at the time of this writting, gitlab.freedesktop.org is under maintenance, but it should be back soon. Thank you! Helen > > Changelog: > > v15:- Moved drm-shmem reference counters to use kref that allows to > optimize unlocked functions, like was suggested by Boris Brezillon. > > - Changed drm/gem/shmem function names to use _locked postfix and > dropped the _unlocked, making the naming scheme consistent across > DRM code, like was suggested by Boris Brezillon. > > - Added patch that fixes UAF in drm-shmem for drivers that import > dma-buf and then release buffer in the import error code path. > > - Added patch that makes drm-shmem use new flag for SGT's get_pages() > refcounting, preventing unbalanced refcounting when GEM is freed. > > - Fixed guest blob pinning in virtio-gpu driver that was missed > previously in the shrinker patch. > > - Moved VC4 and virtio-gpu drivers to use drm_gem_put() in > GEM-creation error code paths, which is now required by drm-shmem > and was missed in a previous patch versions. > > - Virtio-GPU now attaches shmem pages to host on first use and not > when BO is created. In older patch versions there was a potential > race condition in the BO creation code path where both > get_sgt()+object_attach() should've been made under same resv lock, > otherwise pages could be evicted before attachment is invoked. > > - Virtio-GPU and drm-shmem shrinker patches are split into smaller > ones. > > v14:- All the prerequisite reservation locking patches landed upstream, > previously were a part of this series in v13 and older. > > > https://lore.kernel.org/dri-devel/20230529223935.2672495-1-dmitry.osipe...@collabora.com/ > > - Added patches to improve locked/unlocked function names, like was > suggested by Boris Brezillon for v13. > > - Made all exported drm-shmem symbols GPL, like was previously > discussed with Thomas Zimmermann on this series. > > - Improved virtio-gpu shrinker patch. Now it won't detach purged BO > when userspace closes GEM. Crosvm (and not qemu) checks res_id on > CMD_CTX_DETACH_RESOURCE and prints noisy error message if ID is > invalid, which wasn't noticed before. > > v13:- Updated virtio-gpu shrinker patch to use drm_gem_shmem_object_pin() > directly instead of drm_gem_pin() and dropped patch that exported > drm_gem_pin() functions, like was requested by Thomas Zimmermann in > v12. > > v12:- Fixed the "no previous prototype for function" warning reported by > kernel build bot for v11. > > - Fixed the missing reservation lock reported by Intel CI for VGEM > driver. Other drivers using drm-shmem were affected similarly to > VGEM. The problem was in the dma-buf attachment code path that led > to drm-shmem pinning function which assumed the held reservation lock > by drm_gem_pin(). In the past that code path was causing trouble for > i915 driver and we've changed the locking scheme for the attachment > code path in the dma-buf core to let exporters to handle the locking > themselves. After a closer investigation, I realized that my assumption > about testing of dma-buf export code path using Panfrost driver was > incorrect. Now I created additional local test to exrecise the Panfrost > export path. I also reproduced the issue reported by the Intel CI for > v10. It's all fixed now by making the drm_gem_shmem_pin() to take the > resv lock by itself. > > - Patches are based on top of drm-tip, CC'd intel-gfx CI for testing. > > v11:- Rebased on a recent linux-next. Added new patch as a result: > > drm/shmem-helper: Export drm_gem_shmem_get_pages_sgt_locked() > > It's needed by the virtio-gpu driver to swap-in/unevict shmem > object, previously get_pages_sgt() didn't use locking. > > - Separated the "Add memory shrinker" patch into smaller parts to ease > the reviewing, as was requested by Thomas Zimmermann: > > drm/shmem-helper: Factor out pages alloc/release from > drm_gem_shmem_get/put_pages() > drm/shmem-helper: Add pages_pin_count
[Intel-gfx] [PATCH v15 00/23] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers
This series: 1. Adds common drm-shmem memory shrinker 2. Enables shrinker for VirtIO-GPU driver 3. Switches Panfrost driver to the common shrinker Changelog: v15:- Moved drm-shmem reference counters to use kref that allows to optimize unlocked functions, like was suggested by Boris Brezillon. - Changed drm/gem/shmem function names to use _locked postfix and dropped the _unlocked, making the naming scheme consistent across DRM code, like was suggested by Boris Brezillon. - Added patch that fixes UAF in drm-shmem for drivers that import dma-buf and then release buffer in the import error code path. - Added patch that makes drm-shmem use new flag for SGT's get_pages() refcounting, preventing unbalanced refcounting when GEM is freed. - Fixed guest blob pinning in virtio-gpu driver that was missed previously in the shrinker patch. - Moved VC4 and virtio-gpu drivers to use drm_gem_put() in GEM-creation error code paths, which is now required by drm-shmem and was missed in a previous patch versions. - Virtio-GPU now attaches shmem pages to host on first use and not when BO is created. In older patch versions there was a potential race condition in the BO creation code path where both get_sgt()+object_attach() should've been made under same resv lock, otherwise pages could be evicted before attachment is invoked. - Virtio-GPU and drm-shmem shrinker patches are split into smaller ones. v14:- All the prerequisite reservation locking patches landed upstream, previously were a part of this series in v13 and older. https://lore.kernel.org/dri-devel/20230529223935.2672495-1-dmitry.osipe...@collabora.com/ - Added patches to improve locked/unlocked function names, like was suggested by Boris Brezillon for v13. - Made all exported drm-shmem symbols GPL, like was previously discussed with Thomas Zimmermann on this series. - Improved virtio-gpu shrinker patch. Now it won't detach purged BO when userspace closes GEM. Crosvm (and not qemu) checks res_id on CMD_CTX_DETACH_RESOURCE and prints noisy error message if ID is invalid, which wasn't noticed before. v13:- Updated virtio-gpu shrinker patch to use drm_gem_shmem_object_pin() directly instead of drm_gem_pin() and dropped patch that exported drm_gem_pin() functions, like was requested by Thomas Zimmermann in v12. v12:- Fixed the "no previous prototype for function" warning reported by kernel build bot for v11. - Fixed the missing reservation lock reported by Intel CI for VGEM driver. Other drivers using drm-shmem were affected similarly to VGEM. The problem was in the dma-buf attachment code path that led to drm-shmem pinning function which assumed the held reservation lock by drm_gem_pin(). In the past that code path was causing trouble for i915 driver and we've changed the locking scheme for the attachment code path in the dma-buf core to let exporters to handle the locking themselves. After a closer investigation, I realized that my assumption about testing of dma-buf export code path using Panfrost driver was incorrect. Now I created additional local test to exrecise the Panfrost export path. I also reproduced the issue reported by the Intel CI for v10. It's all fixed now by making the drm_gem_shmem_pin() to take the resv lock by itself. - Patches are based on top of drm-tip, CC'd intel-gfx CI for testing. v11:- Rebased on a recent linux-next. Added new patch as a result: drm/shmem-helper: Export drm_gem_shmem_get_pages_sgt_locked() It's needed by the virtio-gpu driver to swap-in/unevict shmem object, previously get_pages_sgt() didn't use locking. - Separated the "Add memory shrinker" patch into smaller parts to ease the reviewing, as was requested by Thomas Zimmermann: drm/shmem-helper: Factor out pages alloc/release from drm_gem_shmem_get/put_pages() drm/shmem-helper: Add pages_pin_count field drm/shmem-helper: Switch drm_gem_shmem_vmap/vunmap to use pin/unpin drm/shmem-helper: Factor out unpinning part from drm_gem_shmem_purge() - Addessed the v10 review comments from Thomas Zimmermann: return errno instead of bool, sort code alphabetically, rename function and etc minor changes. - Added new patch to remove the "map->is_iomem" from drm-shmem, as was suggested by Thomas Zimmermann. - Added acks and r-b's that were given to v10. v10:- Was partially applied to misc-fixes/next. https://lore.kernel.org/dri-devel/6c16f303-81df-7ebe-85e9-51bb40a8b...@collabora.com/T/ Dmitry Osipenko (23): drm/shmem-helper: Fix UAF in error path when freeing SGT of imported GEM drm/shmem-helper: Use flag for tracking page count bumped by get_pages_sgt() drm/gem: Change
