HMM (Heterogeneous Memory Management) v3

2014-06-13 Thread Jérôme Glisse
driver. I however intend to grow the functionalities of the hmm dummy driver in order to make a test and regression suite for the core hmm. Cheers, Jérôme Glisse -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More

[PATCH 1/5] mm: differentiate unmap for vmscan from other unmap.

2014-06-13 Thread Jérôme Glisse
From: Jérôme Glisse jgli...@redhat.com New code will need to be able to differentiate between a regular unmap and an unmap trigger by vmscan in which case we want to be as quick as possible. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- include/linux/rmap.h | 15 --- mm/memory

[PATCH 4/5] hmm: heterogeneous memory management v3

2014-06-13 Thread Jérôme Glisse
From: Jérôme Glisse jgli...@redhat.com Motivation: Heterogeneous memory management is intended to allow a device to transparently access a process address space without having to lock pages of the process or take references on them. In other word mirroring a process address space while allowing

[PATCH 5/5] hmm/dummy: dummy driver to showcase the hmm api v2

2014-06-13 Thread Jérôme Glisse
From: Jérôme Glisse jgli...@redhat.com This is a dummy driver which full fill two purposes : - showcase the hmm api and gives references on how to use it. - provide an extensive user space api to stress test hmm. This is a particularly dangerous module as it allow to access a mirror

[PATCH 3/5] mmu_notifier: pass through vma to invalidate_range and invalidate_page

2014-06-13 Thread Jérôme Glisse
From: Jérôme Glisse jgli...@redhat.com New user of the mmu_notifier interface need to lookup vma in order to perform the invalidation operation. Instead of redoing a vma lookup inside the callback just pass through the vma from the call site where it is already available. This needs small

[PATCH 2/5] mmu_notifier: add action information to address invalidation.

2014-06-13 Thread Jérôme Glisse
From: Jérôme Glisse jgli...@redhat.com The action information will be usefull for new user of mmu_notifier API. The action argument differentiate between a vma disappearing, a page being write protected or simply a page being unmaped. This allow new user to take different action for instance

mm preparatory patches for HMM and IOMMUv2

2014-06-27 Thread Jérôme Glisse
but i intend to send a v4 next week. So i really would like to see those included for next release. As usual comments welcome. Cheers, Jérôme Glisse -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info

[PATCH 4/6] mmu_notifier: pass through vma to invalidate_range and invalidate_page

2014-06-27 Thread Jérôme Glisse
From: Jérôme Glisse jgli...@redhat.com New user of the mmu_notifier interface need to lookup vma in order to perform the invalidation operation. Instead of redoing a vma lookup inside the callback just pass through the vma from the call site where it is already available. This needs small

[PATCH 3/6] mmu_notifier: add event information to address invalidation v2

2014-06-27 Thread Jérôme Glisse
From: Jérôme Glisse jgli...@redhat.com The event information will be usefull for new user of mmu_notifier API. The event argument differentiate between a vma disappearing, a page being write protected or simply a page being unmaped. This allow new user to take different path for different event

[PATCH 2/6] mm: differentiate unmap for vmscan from other unmap.

2014-06-27 Thread Jérôme Glisse
From: Jérôme Glisse jgli...@redhat.com New code will need to be able to differentiate between a regular unmap and an unmap trigger by vmscan in which case we want to be as quick as possible. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- include/linux/rmap.h | 15 --- mm/memory

[PATCH 1/6] mmput: use notifier chain to call subsystem exit handler.

2014-06-27 Thread Jérôme Glisse
From: Jérôme Glisse jgli...@redhat.com Several subsystem require a callback when a mm struct is being destroy so that they can cleanup there respective per mm struct. Instead of having each subsystem add its callback to mmput use a notifier chain to call each of the subsystem. This will allow

HMM (Heterogeneous Memory Management) v10

2015-08-13 Thread Jérôme Glisse
Minor fixes since last post (1), apply on top of 4.2-rc6 done that because conflict in infiniband are harder to solve then conflict with mm tree. Tree with the patchset: git://people.freedesktop.org/~glisse/linux hmm-v10 branch Previous cover letter : HMM (Heterogeneous Memory Management) is

[RFC PATCH 6/8 v2] IB/mlx5/hmm: add mlx5 HMM device initialization and callback v3.

2015-08-13 Thread Jérôme Glisse
This add the core HMM callback for mlx5 device driver and initialize the HMM device for the mlx5 infiniband device driver. Changed since v1: - Adapt to new hmm_mirror lifetime rules. - HMM_ISDIRTY no longer exist. Changed since v2: - Adapt to HMM page table changes. Signed-off-by: Jérôme

[PATCH 04/15] HMM: add new HMM page table flag (select flag).

2015-08-13 Thread Jérôme Glisse
When migrating memory the same array for HMM page table entry might be use with several different devices. Add a new select flag so current device driver callback can know which entry are selected for the device. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- include/linux/hmm_pt.h | 6

[PATCH 05/15] HMM: handle HMM device page table entry on mirror page table fault and update.

2015-08-13 Thread Jérôme Glisse
When faulting or updating the device page table properly handle the case of device memory entry. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- mm/hmm.c | 13 + 1 file changed, 13 insertions(+) diff --git a/mm/hmm.c b/mm/hmm.c index 08c0160..8b1003a 100644 --- a/mm/hmm.c +++ b

[PATCH 07/15] HMM: mm add helper to update page table when migrating memory v2.

2015-08-13 Thread Jérôme Glisse
migrating. - Fix reverse logic on mm_forbids_zeropage() - Add comment on why we add to lru list new page. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- include/linux/mm.h | 14 ++ mm/memory.c| 471 + 2 files changed, 485 insertions

[PATCH 07/15] HMM: add per mirror page table v4.

2015-08-13 Thread Jérôme Glisse
-off-by: Jérôme Glisse jgli...@redhat.com Signed-off-by: Sherry Cheung sche...@nvidia.com Signed-off-by: Subhash Gutti sgu...@nvidia.com Signed-off-by: Mark Hairgrove mhairgr...@nvidia.com Signed-off-by: John Hubbard jhubb...@nvidia.com Signed-off-by: Jatin Kumar jaku...@nvidia.com --- include/linux

[PATCH 14/15] HMM: add documentation explaining HMM internals and how to use it.

2015-08-13 Thread Jérôme Glisse
This add documentation with a high level overview of how HMM works and a more in depth view of how it should be use by device driver writers. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- Documentation/vm/hmm.txt | 219 +++ 1 file changed, 219

[PATCH 15/15] hmm/dummy: dummy driver for testing and showcasing the HMM API

2015-08-13 Thread Jérôme Glisse
file. Hence it should not be enabled by default and only people actively developing for hmm should use it. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- drivers/char/Kconfig | 9 + drivers/char/Makefile | 1 + drivers/char/hmm_dummy.c | 923

[PATCH 12/15] HMM: add dirty range helper (toggle dirty bit inside mirror page table) v2.

2015-08-13 Thread Jérôme Glisse
Device driver must properly toggle the dirty inside the mirror page table so dirtyness is properly accounted when core mm code needs to know. Provide a simple helper to toggle that bit for a range of address. Changed since v1: - Adapt to HMM page table changes. Signed-off-by: Jérôme Glisse

[PATCH 09/15] HMM: add mm page table iterator helpers.

2015-08-13 Thread Jérôme Glisse
map accross call which fit well with the HMM mirror page table update code. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- mm/hmm.c | 95 1 file changed, 95 insertions(+) diff --git a/mm/hmm.c b/mm/hmm.c index 88f7801..f0c3c81

[PATCH 13/15] HMM: DMA map memory on behalf of device driver v2.

2015-08-13 Thread Jérôme Glisse
table changes. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- include/linux/hmm_pt.h | 11 +++ mm/hmm.c | 200 +++-- 2 files changed, 173 insertions(+), 38 deletions(-) diff --git a/include/linux/hmm_pt.h b/include/linux/hmm_pt.h index

[PATCH 11/15] HMM: add discard range helper (to clear and free resources for a range).

2015-08-13 Thread Jérôme Glisse
THE HARDWARE WILL NO LONGER ACCESS THE RANGE BECAUSE CALLING THIS HELPER ! Signed-off-by: Jérôme Glisse jgli...@redhat.com --- include/linux/hmm.h | 3 +++ mm/hmm.c| 24 2 files changed, 27 insertions(+) diff --git a/include/linux/hmm.h b/include/linux/hmm.h index

[PATCH 03/15] mmu_notifier: pass page pointer to mmu_notifier_invalidate_page() v2

2015-08-13 Thread Jérôme Glisse
page table. To avoid complex reverse dma mapping lookup just pass along a pointer to the page being invalidated. Changed since v1: - English syntax fixes. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- drivers/infiniband/core/umem_odp.c | 1 + drivers/iommu/amd_iommu_v2.c | 1

[PATCH 05/15] HMM: introduce heterogeneous memory management v5.

2015-08-13 Thread Jérôme Glisse
. - Export mirror ref/unref functions. - English syntax fixes. Changed since v4: - Properly reference existing hmm struct if any. Signed-off-by: Jérôme Glisse jgli...@redhat.com Signed-off-by: Sherry Cheung sche...@nvidia.com Signed-off-by: Subhash Gutti sgu...@nvidia.com Signed-off-by: Mark Hairgrove

[PATCH 09/15] HMM: allow to get pointer to spinlock protecting a directory.

2015-08-13 Thread Jérôme Glisse
Several use case for getting pointer to spinlock protecting a directory. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- include/linux/hmm_pt.h | 25 + 1 file changed, 25 insertions(+) diff --git a/include/linux/hmm_pt.h b/include/linux/hmm_pt.h index f745d6c

[PATCH 13/15] HMM: CPU page fault on migrated memory.

2015-08-13 Thread Jérôme Glisse
When CPU try to access memory that have been migrated to device memory we have to copy it back to system memory. This patch implement the CPU page fault handler for special HMM pte swap entry. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- mm/hmm.c | 54

[PATCH 06/15] HMM: mm add helper to update page table when migrating memory back v2.

2015-08-13 Thread Jérôme Glisse
change along the way. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- include/linux/mm.h | 12 +++ mm/memory.c| 257 + 2 files changed, 269 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 580fe65

[PATCH 15/15] HMM/dummy: add fake device memory to dummy HMM device driver.

2015-08-13 Thread Jérôme Glisse
This patch add fake device memory by simply using regular system memory page and pretending they are not accessible by the CPU directly. This serve to showcase how migration to device memory can be impemented inside a real device driver. Signed-off-by: Jérôme Glisse jgli...@redhat.com

[PATCH 11/15] HMM: add helpers for migration back to system memory v3.

2015-08-13 Thread Jérôme Glisse
page table changes. - Fix bug in migration failure code path. Signed-off-by: Jérôme Glisse jgli...@redhat.com Signed-off-by: Sherry Cheung sche...@nvidia.com Signed-off-by: Subhash Gutti sgu...@nvidia.com Signed-off-by: Mark Hairgrove mhairgr...@nvidia.com Signed-off-by: John Hubbard jhubb

[PATCH 12/15] HMM: fork copy migrated memory into system memory for child process.

2015-08-13 Thread Jérôme Glisse
When forking if process being fork had any memory migrated to some device memory, we need to make a system copy for the child process. Latter patches can revisit this and use the same COW semantic for device memory. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- mm/hmm.c | 38

[PATCH 08/15] HMM: add device page fault support v4.

2015-08-13 Thread Jérôme Glisse
in hmm_mirror_fault() Changed since v3: - Adapt to HMM page table changes. Signed-off-by: Jérôme Glisse jgli...@redhat.com Signed-off-by: Sherry Cheung sche...@nvidia.com Signed-off-by: Subhash Gutti sgu...@nvidia.com Signed-off-by: Mark Hairgrove mhairgr...@nvidia.com Signed-off-by: John Hubbard jhubb

[PATCH 06/15] HMM: add HMM page table v4.

2015-08-13 Thread Jérôme Glisse
() - hmm_pt_iter_populate(). - Add hmm_pt_iter_walk() - Remove hmm_pt_iter_next() (useless now). - Code simplification and improved comments. - Fix hmm_pt_fini_directory(). Changed since v3: - Fix hmm_pt_iter_directory_unref_safe(). Signed-off-by: Jérôme Glisse jgli...@redhat.com Signed-off

[PATCH 00/15] HMM anonymous memory migration.

2015-08-13 Thread Jérôme Glisse
Minor fixes since last post (1), apply on top of rc6 done that because conflict in infiniband are harder to solve then conflict with mm tree. Tree with the patchset: git://people.freedesktop.org/~glisse/linux hmm-v10 branch This part of the patchset implement anonymous memory migration. It

[PATCH 02/15] HMM: add special swap filetype for memory migrated to device v2.

2015-08-13 Thread Jérôme Glisse
and hooks HMM helpers functions inside the page fault and fork code path. Changed since v1: - Fix name when of HMM CPU page fault function. Signed-off-by: Jérôme Glisse jgli...@redhat.com Signed-off-by: Sherry Cheung sche...@nvidia.com Signed-off-by: Subhash Gutti sgu...@nvidia.com Signed-off

[PATCH 01/15] fork: pass the dst vma to copy_page_range() and its sub-functions.

2015-08-13 Thread Jérôme Glisse
with HMM will have to copy migrated page on fork. We do not expect this to be a common or advised thing to do so we resort to the simpler solution of allocating new page. If this kind of usage turns out to be important we will revisit way to achieve COW even for remote memory. Signed-off-by: Jérôme

[PATCH 03/15] HMM: add new HMM page table flag (valid device memory).

2015-08-13 Thread Jérôme Glisse
For memory migrated to device we need a new type of memory entry. Signed-off-by: Jérôme Glisse jgli...@redhat.com Signed-off-by: Sherry Cheung sche...@nvidia.com Signed-off-by: Subhash Gutti sgu...@nvidia.com Signed-off-by: Mark Hairgrove mhairgr...@nvidia.com Signed-off-by: John Hubbard jhubb

[RFC PATCH 1/8 v2] IB/mlx5: add a new parameter to __mlx_ib_populated_pas for ODP with HMM.

2015-08-13 Thread Jérôme Glisse
When using HMM for ODP it will be useful to pass the current mirror page table iterator for __mlx_ib_populated_pas() function benefit. Add void parameter for this. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- drivers/infiniband/hw/mlx5/mem.c | 8 +--- drivers/infiniband/hw/mlx5

[RFC PATCH 5/8 v2] IB/odp/hmm: add core infiniband structure and helper for ODP with HMM v3.

2015-08-13 Thread Jérôme Glisse
: - Adapt to new hmm_mirror lifetime rules. - Fix scan of existing mirror in ib_umem_odp_get(). Changed since v2: - Remove FIXME for empty umem as it is an invalid case. - Fix HMM version of ib_umem_odp_release() Signed-off-by: Jérôme Glisse jgli...@redhat.com Signed-off-by: John Hubbard

[PATCH 02/15] mmu_notifier: keep track of active invalidation ranges v4

2015-08-13 Thread Jérôme Glisse
are suppose to be use. - English syntax fixes. Signed-off-by: Jérôme Glisse jgli...@redhat.com Reviewed-by: Rik van Riel r...@redhat.com Reviewed-by: Haggai Eran hagg...@mellanox.com --- drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | 13 ++-- drivers/gpu/drm/i915/i915_gem_userptr.c | 10 +-- drivers

[PATCH 10/15] HMM: split DMA mapping function in two.

2015-08-13 Thread Jérôme Glisse
To be able to reuse the DMA mapping logic, split it in two functions. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- mm/hmm.c | 120 ++- 1 file changed, 65 insertions(+), 55 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index

[PATCH 14/15] HMM: add mirror fault support for system to device memory migration v3.

2015-08-13 Thread Jérôme Glisse
for migration, calling mm_hmm_migrate_cleanup() is wrong. Signed-off-by: Jérôme Glisse jgli...@redhat.com Signed-off-by: Sherry Cheung sche...@nvidia.com Signed-off-by: Subhash Gutti sgu...@nvidia.com Signed-off-by: Mark Hairgrove mhairgr...@nvidia.com Signed-off-by: John Hubbard jhubb...@nvidia.com

[PATCH 08/15] HMM: new callback for copying memory from and to device memory v2.

2015-08-13 Thread Jérôme Glisse
From: Jerome Glisse jgli...@redhat.com This patch only adds the new callback device driver must implement to copy memory from and to device memory. Changed since v1: - Pass down the vma to the copy function. Signed-off-by: Jérôme Glisse jgli...@redhat.com Signed-off-by: Sherry Cheung sche

[PATCH 04/15] mmu_notifier: allow range invalidation to exclude a specific mmu_notifier

2015-08-13 Thread Jérôme Glisse
This patch allow to invalidate a range while excluding call to a specific mmu_notifier which allow for a subsystem to invalidate a range for everyone but itself. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- include/linux/mmu_notifier.h | 66 mm

[PATCH 01/15] mmu_notifier: add event information to address invalidation v8

2015-08-13 Thread Jérôme Glisse
. - Differentiate fork from other case. Changed since v7: - Renamed MMU_HUGE_PAGE_SPLIT to MMU_HUGE_PAGE_SPLIT. - Renamed MMU_ISDIRTY to MMU_CLEAR_SOFT_DIRTY. - Renamed MMU_WRITE_PROTECT to MMU_KSM_WRITE_PROTECT. - English syntax fixes. Signed-off-by: Jérôme Glisse jgli...@redhat.com Reviewed

[RFC PATCH 8/8 v2] IB/mlx5/hmm: enable ODP using HMM v2.

2015-08-13 Thread Jérôme Glisse
All pieces are in place for ODP (on demand paging) to work using HMM. Add kernel option and final code to enable it. Changed since v1: - Added kernel option in this last patch of the serie. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- drivers/infiniband/Kconfig | 10

[RFC PATCH 4/8 v2] IB/odp/hmm: prepare for HMM code path.

2015-08-13 Thread Jérôme Glisse
This is a preparatory patch for HMM implementation of ODP (on demand paging). It shuffle codes around that will be share between current ODP implementation and HMM code path. It also convert many #ifdef CONFIG to #if IS_ENABLED(). Signed-off-by: Jérôme Glisse jgli...@redhat.com --- drivers

[RFC PATCH 7/8 v2] IB/mlx5/hmm: add page fault support for ODP on HMM v2.

2015-08-13 Thread Jérôme Glisse
This patch add HMM specific support for hardware page faulting of user memory region. Changed since v1: - Adapt to HMM page table changes. - Turn some sanity test to BUG_ON(). Signed-off-by: Jérôme Glisse jgli...@redhat.com --- drivers/infiniband/hw/mlx5/odp.c | 144

[PATCH 10/15] HMM: use CPU page table during invalidation.

2015-08-13 Thread Jérôme Glisse
by using the page passed by the mmu_notifier layer. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- mm/hmm.c | 53 +++-- 1 file changed, 35 insertions(+), 18 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index f0c3c81..d652233 100644 --- a/mm/hmm.c

[RFC PATCH 2/8 v2] IB/mlx5: add a new parameter to mlx5_ib_update_mtt() for ODP with HMM.

2015-08-13 Thread Jérôme Glisse
When using HMM for ODP it will be useful to pass the current mirror page table iterator for mlx5_ib_update_mtt() function benefit. Add void parameter for this. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- drivers/infiniband/hw/mlx5/mlx5_ib.h | 2 +- drivers/infiniband/hw/mlx5/mr.c

[RFC PATCH 0/8 v2] Implement ODP using HMM v2

2015-08-13 Thread Jérôme Glisse
Posting just for comment, still waiting on HMM to be accepted before this patchset can be considered for inclusion. This patchset implement the on demand paging feature using HMM. It depends on the HMM patchset v10 (previous post (1)). Long term plan is to replace ODP with HMM allowing to share

[RFC PATCH 3/8 v2] IB/odp: export rbt_ib_umem_for_each_in_range()

2015-08-13 Thread Jérôme Glisse
The mlx5 driver will need this function for its driver specific bit of ODP (on demand paging) on HMM (Heterogeneous Memory Management). Signed-off-by: Jérôme Glisse jgli...@redhat.com --- drivers/infiniband/core/umem_rbtree.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/infiniband

[PATCH 11/15] HMM: add discard range helper (to clear and free resources for a range).

2015-07-17 Thread Jérôme Glisse
THE HARDWARE WILL NO LONGER ACCESS THE RANGE BECAUSE CALLING THIS HELPER ! Signed-off-by: Jérôme Glisse jgli...@redhat.com --- include/linux/hmm.h | 3 +++ mm/hmm.c| 24 2 files changed, 27 insertions(+) diff --git a/include/linux/hmm.h b/include/linux/hmm.h index

[PATCH 06/15] HMM: add HMM page table v3.

2015-07-17 Thread Jérôme Glisse
() - hmm_pt_iter_populate(). - Add hmm_pt_iter_walk() - Remove hmm_pt_iter_next() (useless now). - Code simplification and improved comments. - Fix hmm_pt_fini_directory(). Signed-off-by: Jérôme Glisse jgli...@redhat.com Signed-off-by: Sherry Cheung sche...@nvidia.com Signed-off-by: Subhash Gutti

[PATCH 05/15] HMM: introduce heterogeneous memory management v4.

2015-07-17 Thread Jérôme Glisse
. - Export mirror ref/unref functions. - English syntax fixes. Signed-off-by: Jérôme Glisse jgli...@redhat.com Signed-off-by: Sherry Cheung sche...@nvidia.com Signed-off-by: Subhash Gutti sgu...@nvidia.com Signed-off-by: Mark Hairgrove mhairgr...@nvidia.com Signed-off-by: John Hubbard jhubb...@nvidia.com

[PATCH 08/15] HMM: add device page fault support v4.

2015-07-17 Thread Jérôme Glisse
in hmm_mirror_fault() Changed since v3: - Adapt to HMM page table changes. Signed-off-by: Jérôme Glisse jgli...@redhat.com Signed-off-by: Sherry Cheung sche...@nvidia.com Signed-off-by: Subhash Gutti sgu...@nvidia.com Signed-off-by: Mark Hairgrove mhairgr...@nvidia.com Signed-off-by: John Hubbard jhubb

[PATCH 14/15] HMM: add documentation explaining HMM internals and how to use it.

2015-07-17 Thread Jérôme Glisse
This add documentation with a high level overview of how HMM works and a more in depth view of how it should be use by device driver writers. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- Documentation/vm/hmm.txt | 219 +++ 1 file changed, 219

[PATCH 12/15] HMM: add dirty range helper (toggle dirty bit inside mirror page table) v2.

2015-07-17 Thread Jérôme Glisse
Device driver must properly toggle the dirty inside the mirror page table so dirtyness is properly accounted when core mm code needs to know. Provide a simple helper to toggle that bit for a range of address. Changed since v1: - Adapt to HMM page table changes. Signed-off-by: Jérôme Glisse

[PATCH 00/15] HMM (Heterogeneous Memory Management) v9

2015-07-17 Thread Jérôme Glisse
Not much changed since last post (1), i did incorporate comments i got so far, fixed couple bugs here and there and simplified the HMM page table code. I am splitting the patchset into 3 parts. The first part has the core of HMM and is enough for device that do not care about memory migration. The

[PATCH 09/15] HMM: add mm page table iterator helpers.

2015-07-17 Thread Jérôme Glisse
map accross call which fit well with the HMM mirror page table update code. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- mm/hmm.c | 95 1 file changed, 95 insertions(+) diff --git a/mm/hmm.c b/mm/hmm.c index a9e3dc5..826080b

[PATCH 13/15] HMM: DMA map memory on behalf of device driver v2.

2015-07-17 Thread Jérôme Glisse
table changes. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- include/linux/hmm_pt.h | 11 +++ mm/hmm.c | 200 +++-- 2 files changed, 173 insertions(+), 38 deletions(-) diff --git a/include/linux/hmm_pt.h b/include/linux/hmm_pt.h index

[PATCH 15/15] hmm/dummy: dummy driver for testing and showcasing the HMM API

2015-07-17 Thread Jérôme Glisse
file. Hence it should not be enabled by default and only people actively developing for hmm should use it. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- drivers/char/Kconfig | 9 + drivers/char/Makefile | 1 + drivers/char/hmm_dummy.c | 925

[PATCH 07/15] HMM: add per mirror page table v4.

2015-07-17 Thread Jérôme Glisse
-off-by: Jérôme Glisse jgli...@redhat.com Signed-off-by: Sherry Cheung sche...@nvidia.com Signed-off-by: Subhash Gutti sgu...@nvidia.com Signed-off-by: Mark Hairgrove mhairgr...@nvidia.com Signed-off-by: John Hubbard jhubb...@nvidia.com Signed-off-by: Jatin Kumar jaku...@nvidia.com --- include/linux

[PATCH 10/15] HMM: use CPU page table during invalidation.

2015-07-17 Thread Jérôme Glisse
by using the page passed by the mmu_notifier layer. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- mm/hmm.c | 53 +++-- 1 file changed, 35 insertions(+), 18 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index 826080b..0ecc3b0 100644 --- a/mm/hmm.c

[PATCH 04/15] mmu_notifier: allow range invalidation to exclude a specific mmu_notifier

2015-07-17 Thread Jérôme Glisse
This patch allow to invalidate a range while excluding call to a specific mmu_notifier which allow for a subsystem to invalidate a range for everyone but itself. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- include/linux/mmu_notifier.h | 66 mm

[PATCH 03/15] mmu_notifier: pass page pointer to mmu_notifier_invalidate_page() v2

2015-07-17 Thread Jérôme Glisse
page table. To avoid complex reverse dma mapping lookup just pass along a pointer to the page being invalidated. Changed since v1: - English syntax fixes. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- drivers/infiniband/core/umem_odp.c | 1 + drivers/iommu/amd_iommu_v2.c | 1

[PATCH 01/15] mmu_notifier: add event information to address invalidation v8

2015-07-17 Thread Jérôme Glisse
. - Differentiate fork from other case. Changed since v7: - Renamed MMU_HUGE_PAGE_SPLIT to MMU_HUGE_PAGE_SPLIT. - Renamed MMU_ISDIRTY to MMU_CLEAR_SOFT_DIRTY. - Renamed MMU_WRITE_PROTECT to MMU_KSM_WRITE_PROTECT. - English syntax fixes. Signed-off-by: Jérôme Glisse jgli...@redhat.com Reviewed

[PATCH 02/15] mmu_notifier: keep track of active invalidation ranges v4

2015-07-17 Thread Jérôme Glisse
are suppose to be use. - English syntax fixes. Signed-off-by: Jérôme Glisse jgli...@redhat.com Reviewed-by: Rik van Riel r...@redhat.com Reviewed-by: Haggai Eran hagg...@mellanox.com --- drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | 13 ++-- drivers/gpu/drm/i915/i915_gem_userptr.c | 10 +-- drivers

[PATCH 0/8] Add ODP support using HMM

2015-07-17 Thread Jérôme Glisse
This patchset implement the on demand paging feature using HMM. It depends on the HMM patchset v9 (previous post (1)). Long term plan is to replace ODP with HMM allowing to share same code infrastructure accross different class of devices. HMM (Heterogeneous Memory Management) is an helper layer

[PATCH 5/8] IB/odp/hmm: add core infiniband structure and helper for ODP with HMM v2.

2015-07-17 Thread Jérôme Glisse
: - Adapt to new hmm_mirror lifetime rules. - Fix scan of existing mirror in ib_umem_odp_get(). Signed-off-by: Jérôme Glisse jgli...@redhat.com Signed-off-by: John Hubbard jhubb...@nvidia.com Signed-off-by: Haggai Eran hagg...@mellanox.com --- drivers/infiniband/core/umem_odp.c| 150

[PATCH 7/8] IB/mlx5/hmm: add page fault support for ODP on HMM v2.

2015-07-17 Thread Jérôme Glisse
This patch add HMM specific support for hardware page faulting of user memory region. Changed since v1: - Adapt to HMM page table changes. - Turn some sanity test to BUG_ON(). Signed-off-by: Jérôme Glisse jgli...@redhat.com --- drivers/infiniband/hw/mlx5/odp.c | 144

[PATCH 6/8] IB/mlx5/hmm: add mlx5 HMM device initialization and callback v3.

2015-07-17 Thread Jérôme Glisse
This add the core HMM callback for mlx5 device driver and initialize the HMM device for the mlx5 infiniband device driver. Changed since v1: - Adapt to new hmm_mirror lifetime rules. - HMM_ISDIRTY no longer exist. Changed since v2: - Adapt to HMM page table changes. Signed-off-by: Jérôme

[PATCH 8/8] IB/mlx5/hmm: enable ODP using HMM.

2015-07-17 Thread Jérôme Glisse
All pieces are in place for ODP (on demand paging) to work using HMM. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- drivers/infiniband/core/uverbs_cmd.c | 4 drivers/infiniband/hw/mlx5/main.c| 5 - 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/drivers

[PATCH 1/8] IB/mlx5: add a new parameter to __mlx_ib_populated_pas for ODP with HMM.

2015-07-17 Thread Jérôme Glisse
When using HMM for ODP it will be useful to pass the current mirror page table iterator for __mlx_ib_populated_pas() function benefit. Add void parameter for this. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- drivers/infiniband/hw/mlx5/mem.c | 8 +--- drivers/infiniband/hw/mlx5

[PATCH 3/8] IB/odp: export rbt_ib_umem_for_each_in_range()

2015-07-17 Thread Jérôme Glisse
The mlx5 driver will need this function for its driver specific bit of ODP (on demand paging) on HMM (Heterogeneous Memory Management). Signed-off-by: Jérôme Glisse jgli...@redhat.com --- drivers/infiniband/core/umem_rbtree.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/infiniband

[PATCH 2/8] IB/mlx5: add a new parameter to mlx5_ib_update_mtt() for ODP with HMM.

2015-07-17 Thread Jérôme Glisse
When using HMM for ODP it will be useful to pass the current mirror page table iterator for mlx5_ib_update_mtt() function benefit. Add void parameter for this. Signed-off-by: Jérôme Glisse jgli...@redhat.com --- drivers/infiniband/hw/mlx5/mlx5_ib.h | 2 +- drivers/infiniband/hw/mlx5/mr.c

[PATCH 4/8] IB/odp/hmm: add new kernel option to use HMM for ODP.

2015-07-17 Thread Jérôme Glisse
This is a preparatory patch for HMM implementation of ODP (on demand paging). It introduce a new configure option and add proper build time conditional code section. Enabling INFINIBAND_ON_DEMAND_PAGING_HMM will result in build error with this patch. Signed-off-by: Jérôme Glisse jgli

[PATCH v11 04/15] mmu_notifier: allow range invalidation to exclude a specific mmu_notifier

2015-10-21 Thread Jérôme Glisse
This patch allow to invalidate a range while excluding call to a specific mmu_notifier which allow for a subsystem to invalidate a range for everyone but itself. Signed-off-by: Jérôme Glisse <jgli...@redhat.com> --- include/linux/mmu_notifier.

[PATCH v11 08/15] HMM: add device page fault support v6.

2015-10-21 Thread Jérôme Glisse
ult() Changed since v3: - Adapt to HMM page table changes. Changed since v4: - Fix PROT_NONE, ie do not populate from protnone pte. - Fix huge pmd handling (start address may != pmd start address) - Fix missing entry case. Signed-off-by: Jérôme Glisse <jgli...@redhat.com> Signed-off-by: Sher

[PATCH v11 06/15] HMM: add HMM page table v4.

2015-10-21 Thread Jérôme Glisse
ult() -> hmm_pt_iter_populate(). - Add hmm_pt_iter_walk() - Remove hmm_pt_iter_next() (useless now). - Code simplification and improved comments. - Fix hmm_pt_fini_directory(). Changed since v3: - Fix hmm_pt_iter_directory_unref_safe(). Signed-off-by: Jérôme Glisse <jgli...@redhat.com>

[PATCH v11 14/15] HMM: Add support for hugetlb.

2015-10-21 Thread Jérôme Glisse
Support hugetlb vma allmost like other vma. Exception being that we will not support migration of hugetlb memory. Signed-off-by: Jérôme Glisse <jgli...@redhat.com> --- mm/hmm.c | 62 +- 1 file changed, 61 insertions(+), 1 de

[PATCH v11 02/14] HMM: add special swap filetype for memory migrated to device v2.

2015-10-21 Thread Jérôme Glisse
entry and hooks HMM helpers functions inside the page fault and fork code path. Changed since v1: - Fix name when of HMM CPU page fault function. Signed-off-by: Jérôme Glisse <jgli...@redhat.com> Signed-off-by: Sherry Cheung <sche...@nvidia.com> Signed-off-by: Subhash Gutti <sgu...@n

[PATCH v11 08/14] HMM: new callback for copying memory from and to device memory v2.

2015-10-21 Thread Jérôme Glisse
From: Jerome Glisse <jgli...@redhat.com> This patch only adds the new callback device driver must implement to copy memory from and to device memory. Changed since v1: - Pass down the vma to the copy function. Signed-off-by: Jérôme Glisse <jgli...@redhat.com> Signed-off-by: S

[PATCH v11 03/15] mmu_notifier: pass page pointer to mmu_notifier_invalidate_page() v2

2015-10-21 Thread Jérôme Glisse
page table. To avoid complex reverse dma mapping lookup just pass along a pointer to the page being invalidated. Changed since v1: - English syntax fixes. Signed-off-by: Jérôme Glisse <jgli...@redhat.com> --- drivers/infiniband/core/umem_odp.c | 1 + drivers/iommu/amd_iommu_v2.c

[PATCH v11 01/15] mmu_notifier: add event information to address invalidation v8

2015-10-21 Thread Jérôme Glisse
. - Differentiate fork from other case. Changed since v7: - Renamed MMU_HUGE_PAGE_SPLIT to MMU_HUGE_PAGE_SPLIT. - Renamed MMU_ISDIRTY to MMU_CLEAR_SOFT_DIRTY. - Renamed MMU_WRITE_PROTECT to MMU_KSM_WRITE_PROTECT. - English syntax fixes. Signed-off-by: Jérôme Glisse <jgli...@redhat.

[PATCH v11 00/15] HMM (Heterogeneous Memory Management)

2015-10-21 Thread Jérôme Glisse
Minor fixes since last post (1), apply on top of 4.3rc6. Please consider applying. Tree with the patchset: git://people.freedesktop.org/~glisse/linux hmm-v11 branch HMM (HMM (Heterogeneous Memory Management) is an helper layer for device driver, its main features are : - Shadow CPU page table

[PATCH v11 00/14] HMM anomymous memory migration to device memory

2015-10-21 Thread Jérôme Glisse
Minor fixes since last post, apply on top of 4.3rc6. Tree with the patchset: git://people.freedesktop.org/~glisse/linux hmm-v11 branch This patchset implement anonymous memory migration for HMM. See HMM patchset for full description of what is HMM and why doing HMM :

[PATCH v11 04/14] HMM: add new HMM page table flag (select flag).

2015-10-21 Thread Jérôme Glisse
When migrating memory the same array for HMM page table entry might be use with several different devices. Add a new select flag so current device driver callback can know which entry are selected for the device. Signed-off-by: Jérôme Glisse <jgli...@redhat.com> --- include/linux/hmm_pt

[PATCH v11 10/14] HMM: split DMA mapping function in two.

2015-10-21 Thread Jérôme Glisse
To be able to reuse the DMA mapping logic, split it in two functions. Signed-off-by: Jérôme Glisse <jgli...@redhat.com> --- mm/hmm.c | 120 ++- 1 file changed, 65 insertions(+), 55 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c

[PATCH v11 05/14] HMM: handle HMM device page table entry on mirror page table fault and update.

2015-10-21 Thread Jérôme Glisse
When faulting or updating the device page table properly handle the case of device memory entry. Signed-off-by: Jérôme Glisse <jgli...@redhat.com> --- mm/hmm.c | 13 + 1 file changed, 13 insertions(+) diff --git a/mm/hmm.c b/mm/hmm.c index 1c81c68..6224131 100644 --- a/mm

[PATCH v11 12/14] HMM: fork copy migrated memory into system memory for child process.

2015-10-21 Thread Jérôme Glisse
When forking if process being fork had any memory migrated to some device memory, we need to make a system copy for the child process. Latter patches can revisit this and use the same COW semantic for device memory. Signed-off-by: Jérôme Glisse <jgli...@redhat.com> --- mm/hmm.

[PATCH v11 13/14] HMM: CPU page fault on migrated memory.

2015-10-21 Thread Jérôme Glisse
When CPU try to access memory that have been migrated to device memory we have to copy it back to system memory. This patch implement the CPU page fault handler for special HMM pte swap entry. Signed-off-by: Jérôme Glisse <jgli...@redhat.com> --- mm/hmm.

[PATCH v11 13/15] HMM: DMA map memory on behalf of device driver v2.

2015-10-21 Thread Jérôme Glisse
table changes. Signed-off-by: Jérôme Glisse <jgli...@redhat.com> --- include/linux/hmm_pt.h | 11 +++ mm/hmm.c | 202 +++-- 2 files changed, 174 insertions(+), 39 deletions(-) diff --git a/include/linux/hmm_pt.h b/include/linux/hm

[PATCH v11 15/15] HMM: add documentation explaining HMM internals and how to use it.

2015-10-21 Thread Jérôme Glisse
This add documentation on how HMM works and a more in depth view of how it should be use by device driver writers. Signed-off-by: Jérôme Glisse <jgli...@redhat.com> --- Documentation/vm/hmm.txt | 219 +++ 1 file changed, 219 insertions(+) creat

[PATCH v11 12/15] HMM: add dirty range helper (toggle dirty bit inside mirror page table) v2.

2015-10-21 Thread Jérôme Glisse
Device driver must properly toggle the dirty inside the mirror page table so dirtyness is properly accounted when core mm code needs to know. Provide a simple helper to toggle that bit for a range of address. Changed since v1: - Adapt to HMM page table changes. Signed-off-by: Jérôme Glisse

[PATCH v11 09/15] HMM: add mm page table iterator helpers.

2015-10-21 Thread Jérôme Glisse
map accross call which fit well with the HMM mirror page table update code. Signed-off-by: Jérôme Glisse <jgli...@redhat.com> --- mm/hmm.c | 95 1 file changed, 95 insertions(+) diff --git a/mm/hmm.c b/mm/hmm.c index c

[PATCH v11 11/15] HMM: add discard range helper (to clear and free resources for a range).

2015-10-21 Thread Jérôme Glisse
THE HARDWARE WILL NO LONGER ACCESS THE RANGE BECAUSE CALLING THIS HELPER ! Signed-off-by: Jérôme Glisse <jgli...@redhat.com> --- include/linux/hmm.h | 3 +++ mm/hmm.c| 24 2 files changed, 27 insertions(+) diff --git a/include/linux/hmm.h b/include/linux

[PATCH v11 14/14] HMM: add mirror fault support for system to device memory migration v3.

2015-10-21 Thread Jérôme Glisse
for migration, calling mm_hmm_migrate_cleanup() is wrong. Signed-off-by: Jérôme Glisse <jgli...@redhat.com> Signed-off-by: Sherry Cheung <sche...@nvidia.com> Signed-off-by: Subhash Gutti <sgu...@nvidia.com> Signed-off-by: Mark Hairgrove <mhairgr...@nvidia.com> Signed-off

[PATCH v11 03/14] HMM: add new HMM page table flag (valid device memory).

2015-10-21 Thread Jérôme Glisse
For memory migrated to device we need a new type of memory entry. Signed-off-by: Jérôme Glisse <jgli...@redhat.com> Signed-off-by: Sherry Cheung <sche...@nvidia.com> Signed-off-by: Subhash Gutti <sgu...@nvidia.com> Signed-off-by: Mark Hairgrove <mhairgr...@nvidia.com> Sig

[PATCH v11 05/15] HMM: introduce heterogeneous memory management v5.

2015-10-21 Thread Jérôme Glisse
. - Export mirror ref/unref functions. - English syntax fixes. Changed since v4: - Properly reference existing hmm struct if any. Signed-off-by: Jérôme Glisse <jgli...@redhat.com> Signed-off-by: Sherry Cheung <sche...@nvidia.com> Signed-off-by: Subhash Gutti <sgu...@nvidia.com> S

  1   2   3   4   5   6   7   8   9   >