From: Jérôme Glisse
In __lock_release() we are removing one entry from the stack and
rebuilding the hash chain by re-adding entry above the entry we
just removed. If the entry removed was between 2 entry of same
class then this 2 entry might be coalesced into one single entry
which in turns
From: Jérôme Glisse
In __lock_release() we are removing one entry from the stack and
rebuilding the hash chain by re-adding entry above the entry we
just removed. If the entry removed was between 2 entry of same
class then this 2 entry might be coalesced into one single entry
From: Jérôme Glisse
It can be rather tedious to find why userspace is failing when only
thing kernel report is -ENOMEM. This add an error message so that
user can figure out why they are getting -ENOMEM.
Signed-off-by: Jérôme Glisse
cc:
Cc: Haggai Eran
Cc: Sagi Grimberg
Cc: Shachar Raindel
From: Jérôme Glisse
It can be rather tedious to find why userspace is failing when only
thing kernel report is -ENOMEM. This add an error message so that
user can figure out why they are getting -ENOMEM.
Signed-off-by: Jérôme Glisse
cc:
From: Jérôme Glisse
Fix amd_iommu_detect() to return positive value on success, like
intended, and not zero. This will not change anything in the end
as AMD IOMMU disable swiotlb and properly associate itself with
devices even if detect() doesn't return a positive value.
Signed-off-by: Jérôme
From: Jérôme Glisse
Fix amd_iommu_detect() to return positive value on success, like
intended, and not zero. This will not change anything in the end
as AMD IOMMU disable swiotlb and properly associate itself with
devices even if detect() doesn't return a positive value.
So sorry had to resend because i stupidly forgot to cc mailing list.
Ignore private send done before.
HMM (Heterogeneous Memory Management) is an helper layer for device
that want to mirror a process address space into their own mmu. Main
target is GPU but other hardware, like network device
From: Jérôme Glisse
This patch add the per mirror page table. It also propagate CPU page
table update to this per mirror page table using mmu_notifier callback.
All update are contextualized with an HMM event structure that convey
all information needed by device driver to take proper actions
From: Jérôme Glisse
This patch allow to invalidate a range while excluding call to a specific
mmu_notifier which allow for a subsystem to invalidate a range for everyone
but itself.
Signed-off-by: Jérôme Glisse
---
include/linux/mmu_notifier.h | 60 +++-
From: Jerome Glisse
Once we store the dma mapping inside the secondary page table we can
no longer easily find back the page backing an address. Instead use
the cpu page table which still has the proper informations, except for
the invalidate_page() case which is handled by using the page passed
From: Jérôme Glisse
Listener of mm event might not have easy way to get the struct page
behind and address invalidated with mmu_notifier_invalidate_page()
function as this happens after the cpu page table have been clear/
updated. This happens for instance if the listener is storing a dma
From: Jérôme Glisse
This patch add helper for device page fault. Device page fault helper will
fill the mirror page table using the CPU page table all this synchronized
with any update to CPU page table.
Changed since v1:
- Add comment about directory lock.
Changed since v2:
- Check for
From: Jérôme Glisse
Do the DMA mapping on behalf of the device as HMM is a good place
to perform this common task. Moreover in the future we hope to
add new infrastructure that would make DMA mapping more efficient
(lower overhead per page) by leveraging HMM data structure.
Signed-off-by:
From: Jérôme Glisse
For HMM we will need to resort to the old way of allocating new page
for anonymous memory when that anonymous memory have been migrated
to device memory.
This does not impact any process that do not use HMM through some
device driver. Only process that migrate anonymous
From: Jérôme Glisse
A common use case is for device driver to stop caring for a range of address
long before said range is munmapped by userspace program. To avoid keeping
track of such range provide an helper function that will free HMM resources
for a range of address.
NOTE THAT DEVICE DRIVER
From: Jérôme Glisse
Usefull for HMM when trying to uncharge freshly allocated anonymous
pages after error inside migration memory migration path.
Signed-off-by: Jérôme Glisse
---
include/linux/memcontrol.h | 7 +++
mm/memcontrol.c| 3 ++-
2 files changed, 9 insertions(+), 1
From: Jérôme Glisse
When migrating memory the same array for HMM page table entry might be
use with several different devices. Add a new select flag so current
device driver callback can know which entry are selected for the device.
Signed-off-by: Jérôme Glisse
---
include/linux/hmm_pt.h | 6
From: Jérôme Glisse
When faulting or updating the device page table properly handle the case of
device
memory entry.
Signed-off-by: Jérôme Glisse
---
mm/hmm.c | 14 ++
1 file changed, 14 insertions(+)
diff --git a/mm/hmm.c b/mm/hmm.c
index 761905a..e4585b7 100644
--- a/mm/hmm.c
From: Jerome Glisse
When migrating anonymous memory from system memory to device memory
CPU pte are replaced with special HMM swap entry so that page fault,
get user page (gup), fork, ... are properly redirected to HMM helpers.
This patch only add the new swap type entry and hooks HMM helpers
From: Jérôme Glisse
Because inside the mmu_notifier callback we do not have access to the
vma nor do we know which lock we are holding (the mmap semaphore or
the i_mmap_lock) we can not rely on the regular page table walk (nor
do we want as we have to be carefull to not split huge page).
So
From: Jérôme Glisse
For memory migrated to device we need a new type of memory entry.
Signed-off-by: Jérôme Glisse
Signed-off-by: Sherry Cheung
Signed-off-by: Subhash Gutti
Signed-off-by: Mark Hairgrove
Signed-off-by: John Hubbard
Signed-off-by: Jatin Kumar
---
include/linux/hmm_pt.h |
From: Jérôme Glisse
Device driver must properly toggle the dirty inside the mirror page table so
dirtyness is properly accounted when core mm code needs to know. Provide a
simple helper to toggle that bit for a range of address.
Signed-off-by: Jérôme Glisse
---
include/linux/hmm.h | 3 +++
From: Jérôme Glisse
The mmu_notifier_invalidate_range_start() and
mmu_notifier_invalidate_range_end()
can be considered as forming an "atomic" section for the cpu page table update
point of view. Between this two function the cpu page table content is
unreliable
for the address range being
From: Jérôme Glisse
This patch only introduce core HMM functions for registering a new
mirror and stopping a mirror as well as HMM device registering and
unregistering.
The lifecycle of HMM object is handled differently then the one of
mmu_notifier because unlike mmu_notifier there can be
From: Jérôme Glisse
Heterogeneous memory management main purpose is to mirror a process address.
To do so it must maintain a secondary page table that is use by the device
driver to program the device or build a device specific page table.
Radix tree can not be use to create this secondary page
From: Jérôme Glisse
The event information will be useful for new user of mmu_notifier API.
The event argument differentiate between a vma disappearing, a page
being write protected or simply a page being unmaped. This allow new
user to take different path for different event for instance on
From: Jérôme Glisse jgli...@redhat.com
This patch add helper for device page fault. Device page fault helper will
fill the mirror page table using the CPU page table all this synchronized
with any update to CPU page table.
Changed since v1:
- Add comment about directory lock.
Changed since
From: Jérôme Glisse jgli...@redhat.com
Listener of mm event might not have easy way to get the struct page
behind and address invalidated with mmu_notifier_invalidate_page()
function as this happens after the cpu page table have been clear/
updated. This happens for instance if the listener is
From: Jerome Glisse jgli...@redhat.com
Once we store the dma mapping inside the secondary page table we can
no longer easily find back the page backing an address. Instead use
the cpu page table which still has the proper informations, except for
the invalidate_page() case which is handled by
From: Jérôme Glisse jgli...@redhat.com
This patch allow to invalidate a range while excluding call to a specific
mmu_notifier which allow for a subsystem to invalidate a range for everyone
but itself.
Signed-off-by: Jérôme Glisse jgli...@redhat.com
---
include/linux/mmu_notifier.h | 60
So sorry had to resend because i stupidly forgot to cc mailing list.
Ignore private send done before.
HMM (Heterogeneous Memory Management) is an helper layer for device
that want to mirror a process address space into their own mmu. Main
target is GPU but other hardware, like network device
From: Jérôme Glisse jgli...@redhat.com
This patch add the per mirror page table. It also propagate CPU page
table update to this per mirror page table using mmu_notifier callback.
All update are contextualized with an HMM event structure that convey
all information needed by device driver to take
From: Jérôme Glisse jgli...@redhat.com
Because inside the mmu_notifier callback we do not have access to the
vma nor do we know which lock we are holding (the mmap semaphore or
the i_mmap_lock) we can not rely on the regular page table walk (nor
do we want as we have to be carefull to not split
From: Jérôme Glisse jgli...@redhat.com
Usefull for HMM when trying to uncharge freshly allocated anonymous
pages after error inside migration memory migration path.
Signed-off-by: Jérôme Glisse jgli...@redhat.com
---
include/linux/memcontrol.h | 7 +++
mm/memcontrol.c| 3 ++-
2
From: Jérôme Glisse jgli...@redhat.com
When migrating memory the same array for HMM page table entry might be
use with several different devices. Add a new select flag so current
device driver callback can know which entry are selected for the device.
Signed-off-by: Jérôme Glisse
From: Jérôme Glisse jgli...@redhat.com
Do the DMA mapping on behalf of the device as HMM is a good place
to perform this common task. Moreover in the future we hope to
add new infrastructure that would make DMA mapping more efficient
(lower overhead per page) by leveraging HMM data structure.
From: Jérôme Glisse jgli...@redhat.com
For HMM we will need to resort to the old way of allocating new page
for anonymous memory when that anonymous memory have been migrated
to device memory.
This does not impact any process that do not use HMM through some
device driver. Only process that
From: Jérôme Glisse jgli...@redhat.com
A common use case is for device driver to stop caring for a range of address
long before said range is munmapped by userspace program. To avoid keeping
track of such range provide an helper function that will free HMM resources
for a range of address.
NOTE
From: Jérôme Glisse jgli...@redhat.com
When faulting or updating the device page table properly handle the case of
device
memory entry.
Signed-off-by: Jérôme Glisse jgli...@redhat.com
---
mm/hmm.c | 14 ++
1 file changed, 14 insertions(+)
diff --git a/mm/hmm.c b/mm/hmm.c
index
From: Jerome Glisse jgli...@redhat.com
When migrating anonymous memory from system memory to device memory
CPU pte are replaced with special HMM swap entry so that page fault,
get user page (gup), fork, ... are properly redirected to HMM helpers.
This patch only add the new swap type entry and
From: Jérôme Glisse jgli...@redhat.com
This patch only introduce core HMM functions for registering a new
mirror and stopping a mirror as well as HMM device registering and
unregistering.
The lifecycle of HMM object is handled differently then the one of
mmu_notifier because unlike mmu_notifier
From: Jérôme Glisse jgli...@redhat.com
Heterogeneous memory management main purpose is to mirror a process address.
To do so it must maintain a secondary page table that is use by the device
driver to program the device or build a device specific page table.
Radix tree can not be use to create
From: Jérôme Glisse jgli...@redhat.com
The event information will be useful for new user of mmu_notifier API.
The event argument differentiate between a vma disappearing, a page
being write protected or simply a page being unmaped. This allow new
user to take different path for different event
From: Jérôme Glisse jgli...@redhat.com
The mmu_notifier_invalidate_range_start() and
mmu_notifier_invalidate_range_end()
can be considered as forming an atomic section for the cpu page table update
point of view. Between this two function the cpu page table content is
unreliable
for the address
From: Jérôme Glisse jgli...@redhat.com
For memory migrated to device we need a new type of memory entry.
Signed-off-by: Jérôme Glisse jgli...@redhat.com
Signed-off-by: Sherry Cheung sche...@nvidia.com
Signed-off-by: Subhash Gutti sgu...@nvidia.com
Signed-off-by: Mark Hairgrove
From: Jérôme Glisse jgli...@redhat.com
Device driver must properly toggle the dirty inside the mirror page table so
dirtyness is properly accounted when core mm code needs to know. Provide a
simple helper to toggle that bit for a range of address.
Signed-off-by: Jérôme Glisse jgli...@redhat.com
From: Jérôme Glisse
Heterogeneous memory management main purpose is to mirror a process address.
To do so it must maintain a secondary page table that is use by the device
driver to program the device or build a device specific page table.
Radix tree can not be use to create this secondary page
So a resend with corrections base on Haggai comments. This patchset is just
the ground foundation on to which we want to build our features set. Main
feature being migrating memory to device memory. The very first version of
this patchset already show cased proof of concept of much of the
From: Jérôme Glisse
The mmu_notifier_invalidate_range_start() and
mmu_notifier_invalidate_range_end()
can be considered as forming an "atomic" section for the cpu page table update
point of view. Between this two function the cpu page table content is
unreliable
for the address range being
From: Jérôme Glisse
This patch add helper for device page fault. Device page fault helper will
fill the mirror page table using the CPU page table all this synchronized
with any update to CPU page table.
Signed-off-by: Jérôme Glisse
Signed-off-by: Sherry Cheung
Signed-off-by: Subhash Gutti
From: Jérôme Glisse
This patch add the per mirror page table. It also propagate CPU page table
update to this per mirror page table using mmu_notifier callback. All update
are contextualized with an HMM event structure that convey all information
needed by device driver to take proper actions
From: Jérôme Glisse
This patch only introduce core HMM functions for registering a new mirror and
stopping a mirror as well as registering and unregistering a device.
The lifecycle of HMM object is handled differently then one of mmu_notifier
because unlike mmu_notifier there can be concurrent
From: Jérôme Glisse
The event information will be useful for new user of mmu_notifier API.
The event argument differentiate between a vma disappearing, a page
being write protected or simply a page being unmaped. This allow new
user to take different path for different event for instance on
From: Jérôme Glisse jgli...@redhat.com
Heterogeneous memory management main purpose is to mirror a process address.
To do so it must maintain a secondary page table that is use by the device
driver to program the device or build a device specific page table.
Radix tree can not be use to create
From: Jérôme Glisse jgli...@redhat.com
This patch add helper for device page fault. Device page fault helper will
fill the mirror page table using the CPU page table all this synchronized
with any update to CPU page table.
Signed-off-by: Jérôme Glisse jgli...@redhat.com
Signed-off-by: Sherry
From: Jérôme Glisse jgli...@redhat.com
This patch add the per mirror page table. It also propagate CPU page table
update to this per mirror page table using mmu_notifier callback. All update
are contextualized with an HMM event structure that convey all information
needed by device driver to take
From: Jérôme Glisse jgli...@redhat.com
This patch only introduce core HMM functions for registering a new mirror and
stopping a mirror as well as registering and unregistering a device.
The lifecycle of HMM object is handled differently then one of mmu_notifier
because unlike mmu_notifier there
From: Jérôme Glisse jgli...@redhat.com
The mmu_notifier_invalidate_range_start() and
mmu_notifier_invalidate_range_end()
can be considered as forming an atomic section for the cpu page table update
point of view. Between this two function the cpu page table content is
unreliable
for the address
So a resend with corrections base on Haggai comments. This patchset is just
the ground foundation on to which we want to build our features set. Main
feature being migrating memory to device memory. The very first version of
this patchset already show cased proof of concept of much of the
From: Jérôme Glisse jgli...@redhat.com
The event information will be useful for new user of mmu_notifier API.
The event argument differentiate between a vma disappearing, a page
being write protected or simply a page being unmaped. This allow new
user to take different path for different event
From: Jérôme Glisse
The mmu_notifier_invalidate_range_start() and
mmu_notifier_invalidate_range_end()
can be considered as forming an "atomic" section for the cpu page table update
point of view. Between this two function the cpu page table content is
unreliable
for the address range being
From: Jérôme Glisse
This patch add the per mirror page table. It also propagate CPU page table
update to this per mirror page table using mmu_notifier callback. All update
are contextualized with an HMM event structure that convey all information
needed by device driver to take proper actions
From: Jérôme Glisse
Heterogeneous memory management main purpose is to mirror a process address.
To do so it must maintain a secondary page table that is use by the device
driver to program the device or build a device specific page table.
Radix tree can not be use to create this secondary page
From: Jérôme Glisse
This patch add helper for device page fault. Device page fault helper will
fill the mirror page table using the CPU page table all this synchronized
with any update to CPU page table.
Signed-off-by: Jérôme Glisse
Signed-off-by: Sherry Cheung
Signed-off-by: Subhash Gutti
From: Jérôme Glisse
This patch only introduce core HMM functions for registering a new mirror and
stopping a mirror as well as registering and unregistering a device.
The lifecycle of HMM object is handled differently then one of mmu_notifier
because unlike mmu_notifier there can be concurrent
From: Jérôme Glisse
The event information will be useful for new user of mmu_notifier API.
The event argument differentiate between a vma disappearing, a page
being write protected or simply a page being unmaped. This allow new
user to take different path for different event for instance on
So after PTO and before end of year frenzy here is an updated HMM patchset.
While not reusing Linus page table design, i use something that is, in my
view at least, close to it. Also i avoid pretending that this will be useful
to other and move it to hmm specific code. There is a longer
So after PTO and before end of year frenzy here is an updated HMM patchset.
While not reusing Linus page table design, i use something that is, in my
view at least, close to it. Also i avoid pretending that this will be useful
to other and move it to hmm specific code. There is a longer
From: Jérôme Glisse jgli...@redhat.com
This patch only introduce core HMM functions for registering a new mirror and
stopping a mirror as well as registering and unregistering a device.
The lifecycle of HMM object is handled differently then one of mmu_notifier
because unlike mmu_notifier there
From: Jérôme Glisse jgli...@redhat.com
The event information will be useful for new user of mmu_notifier API.
The event argument differentiate between a vma disappearing, a page
being write protected or simply a page being unmaped. This allow new
user to take different path for different event
From: Jérôme Glisse jgli...@redhat.com
This patch add helper for device page fault. Device page fault helper will
fill the mirror page table using the CPU page table all this synchronized
with any update to CPU page table.
Signed-off-by: Jérôme Glisse jgli...@redhat.com
Signed-off-by: Sherry
From: Jérôme Glisse jgli...@redhat.com
Heterogeneous memory management main purpose is to mirror a process address.
To do so it must maintain a secondary page table that is use by the device
driver to program the device or build a device specific page table.
Radix tree can not be use to create
From: Jérôme Glisse jgli...@redhat.com
This patch add the per mirror page table. It also propagate CPU page table
update to this per mirror page table using mmu_notifier callback. All update
are contextualized with an HMM event structure that convey all information
needed by device driver to take
From: Jérôme Glisse jgli...@redhat.com
The mmu_notifier_invalidate_range_start() and
mmu_notifier_invalidate_range_end()
can be considered as forming an atomic section for the cpu page table update
point of view. Between this two function the cpu page table content is
unreliable
for the address
From: Jérôme Glisse
When doing make binrpm-pkg we only want to build the binary and header
package as the documentation of binrpm-pkg target claims. Hence this
patch avoid building the source and devel package. This makes binrpm-pkg
target lot faster and way more usefull.
Changed since v3 :
-
From: Jérôme Glisse jgli...@redhat.com
When doing make binrpm-pkg we only want to build the binary and header
package as the documentation of binrpm-pkg target claims. Hence this
patch avoid building the source and devel package. This makes binrpm-pkg
target lot faster and way more usefull.
From: Jérôme Glisse
When doing make binrpm-pkg we only want to build the binary and header
package as the documentation of binrpm-pkg target claims. Hence this
patch avoid building the source and devel package. This makes binrpm-pkg
target lot faster and way more usefull.
Changed since v2 :
-
From: Jérôme Glisse jgli...@redhat.com
When doing make binrpm-pkg we only want to build the binary and header
package as the documentation of binrpm-pkg target claims. Hence this
patch avoid building the source and devel package. This makes binrpm-pkg
target lot faster and way more usefull.
From: Jérôme Glisse
When doing make binrpm-pkg we do not want to build the source tar
file. This patch avoid doing the gigantic tar file.
Changed since v1
- Avoid building the devel package too.
Signed-off-by: Jérôme Glisse
---
scripts/package/mkspec | 4
1 file changed, 4
From: Jérôme Glisse jgli...@redhat.com
When doing make binrpm-pkg we do not want to build the source tar
file. This patch avoid doing the gigantic tar file.
Changed since v1
- Avoid building the devel package too.
Signed-off-by: Jérôme Glisse jgli...@redhat.com
---
scripts/package/mkspec | 4
From: Jérôme Glisse
Page table is a common structure format most notably use by cpu mmu. The
arch depend page table code has strong tie to the architecture which makes
it unsuitable to be use by other non arch specific code.
This patch implement a generic and arch independent page table. It is
From: Jérôme Glisse
The event information will be usefull for new user of mmu_notifier API.
The event argument differentiate between a vma disappearing, a page
being write protected or simply a page being unmaped. This allow new
user to take different path for different event for instance on
From: Jérôme Glisse
Motivation:
Heterogeneous memory management is intended to allow a device to transparently
access a process address space without having to lock pages of the process or
take references on them. In other word mirroring a process address space while
allowing the regular memory
From: Jérôme Glisse
The mmu_notifier_invalidate_range_start() and
mmu_notifier_invalidate_range_end()
can be considered as forming an "atomic" section for the cpu page table update
point of view. Between this two function the cpu page table content is
unreliable
for the address range being
Andrew so resending with review and ack from Riek and couple minor fixes
along the way. Is there anything blocking this from getting in next kernel ?
Again hardware is coming and there is still a long list of features waiting
on this core set of patches getting in. I reinclude part of my previous
Andrew so resending with review and ack from Riek and couple minor fixes
along the way. Is there anything blocking this from getting in next kernel ?
Again hardware is coming and there is still a long list of features waiting
on this core set of patches getting in. I reinclude part of my previous
From: Jérôme Glisse jgli...@redhat.com
The mmu_notifier_invalidate_range_start() and
mmu_notifier_invalidate_range_end()
can be considered as forming an atomic section for the cpu page table update
point of view. Between this two function the cpu page table content is
unreliable
for the address
From: Jérôme Glisse jgli...@redhat.com
Motivation:
Heterogeneous memory management is intended to allow a device to transparently
access a process address space without having to lock pages of the process or
take references on them. In other word mirroring a process address space while
allowing
From: Jérôme Glisse jgli...@redhat.com
The event information will be usefull for new user of mmu_notifier API.
The event argument differentiate between a vma disappearing, a page
being write protected or simply a page being unmaped. This allow new
user to take different path for different event
From: Jérôme Glisse jgli...@redhat.com
Page table is a common structure format most notably use by cpu mmu. The
arch depend page table code has strong tie to the architecture which makes
it unsuitable to be use by other non arch specific code.
This patch implement a generic and arch independent
From: Jérôme Glisse
Motivation:
Heterogeneous memory management is intended to allow a device to transparently
access a process address space without having to lock pages of the process or
take references on them. In other word mirroring a process address space while
allowing the regular memory
From: Jérôme Glisse
Page table is a common structure format most notably use by cpu mmu. The
arch depend page table code has strong tie to the architecture which makes
it unsuitable to be use by other non arch specific code.
This patch implement a generic and arch independent page table. It is
From: Jérôme Glisse
The event information will be usefull for new user of mmu_notifier API.
The event argument differentiate between a vma disappearing, a page
being write protected or simply a page being unmaped. This allow new
user to take different path for different event for instance on
From: Jérôme Glisse
The mmu_notifier_invalidate_range_start() and
mmu_notifier_invalidate_range_end()
can be considered as forming an "atomic" section for the cpu page table update
point of view. Between this two function the cpu page table content is
unreliable
for the address range being
From: Jérôme Glisse
This is a dummy driver which full fill two purposes :
- showcase the hmm api and gives references on how to use it.
- provide an extensive user space api to stress test hmm.
This is a particularly dangerous module as it allow to access a
mirror of a process address space
Andrew i received no feedback since last time i sent this patchset, so i
would really like to have it merge for next kernel. While right now there
is no kernel driver that leverage this code, the hardware is coming and we
still have a long way to go before we have all the features needed. Right
Andrew i received no feedback since last time i sent this patchset, so i
would really like to have it merge for next kernel. While right now there
is no kernel driver that leverage this code, the hardware is coming and we
still have a long way to go before we have all the features needed. Right
From: Jérôme Glisse jgli...@redhat.com
This is a dummy driver which full fill two purposes :
- showcase the hmm api and gives references on how to use it.
- provide an extensive user space api to stress test hmm.
This is a particularly dangerous module as it allow to access a
mirror of a
From: Jérôme Glisse jgli...@redhat.com
The event information will be usefull for new user of mmu_notifier API.
The event argument differentiate between a vma disappearing, a page
being write protected or simply a page being unmaped. This allow new
user to take different path for different event
From: Jérôme Glisse jgli...@redhat.com
The mmu_notifier_invalidate_range_start() and
mmu_notifier_invalidate_range_end()
can be considered as forming an atomic section for the cpu page table update
point of view. Between this two function the cpu page table content is
unreliable
for the address
1 - 100 of 156 matches
Mail list logo