Re: [PATCH 0/9] Add support for SVM atomics in Nouveau

2021-02-17 Thread Alistair Popple
On Thursday, 11 February 2021 6:55:10 PM AEDT Christoph Hellwig wrote: > On Wed, Feb 10, 2021 at 01:59:13PM -0400, Jason Gunthorpe wrote: > > Really what you want to do here is leave the CPU page in the VMA and > > the page tables where it started and deny CPU access to the page. Then > > all the

Re: [PATCH 0/9] Add support for SVM atomics in Nouveau

2021-02-11 Thread Christoph Hellwig
On Wed, Feb 10, 2021 at 01:59:13PM -0400, Jason Gunthorpe wrote: > Really what you want to do here is leave the CPU page in the VMA and > the page tables where it started and deny CPU access to the page. Then > all the proper machinery will continue to work. > > IMHO "migration" is the wrong idea

Re: [PATCH 0/9] Add support for SVM atomics in Nouveau

2021-02-10 Thread John Hubbard
On 2/10/21 4:59 AM, Daniel Vetter wrote: ... GPU atomic operations to sysmem are hard to categorize, because because application programmers could easily write programs that do a long series of atomic operations. Such a program would be a little weird, but it's hard to rule out. Yeah, but we

Re: [PATCH 0/9] Add support for SVM atomics in Nouveau

2021-02-10 Thread Jason Gunthorpe
On Tue, Feb 09, 2021 at 12:53:27PM -0800, John Hubbard wrote: > This direction sounds at least...possible. Using MMU notifiers instead of pins > is definitely appealing. I'm not quite clear on the callback idea above, but > overall it seems like taking advantage of the ZONE_DEVICE tracking of

Re: [PATCH 0/9] Add support for SVM atomics in Nouveau

2021-02-10 Thread Jason Gunthorpe
On Tue, Feb 09, 2021 at 04:17:38PM -0500, Jerome Glisse wrote: > > > Yes, I would like to avoid the long term pin constraints as well if > > > possible I > > > just haven't found a solution yet. Are you suggesting it might be > > > possible to > > > add a callback in the page migration logic

Re: [PATCH 0/9] Add support for SVM atomics in Nouveau

2021-02-10 Thread Daniel Vetter
On Tue, Feb 09, 2021 at 12:53:27PM -0800, John Hubbard wrote: > On 2/9/21 5:37 AM, Daniel Vetter wrote: > > On Tue, Feb 9, 2021 at 1:57 PM Alistair Popple wrote: > > > > > > On Tuesday, 9 February 2021 9:27:05 PM AEDT Daniel Vetter wrote: > > > > > > > > > > Recent changes to pin_user_pages()

Re: [PATCH 0/9] Add support for SVM atomics in Nouveau

2021-02-09 Thread Jerome Glisse
On Tue, Feb 09, 2021 at 09:35:20AM -0400, Jason Gunthorpe wrote: > On Tue, Feb 09, 2021 at 11:57:28PM +1100, Alistair Popple wrote: > > On Tuesday, 9 February 2021 9:27:05 PM AEDT Daniel Vetter wrote: > > > > > > > > Recent changes to pin_user_pages() prevent the creation of pinned pages > > > >

Re: [PATCH 0/9] Add support for SVM atomics in Nouveau

2021-02-09 Thread John Hubbard
On 2/9/21 5:37 AM, Daniel Vetter wrote: On Tue, Feb 9, 2021 at 1:57 PM Alistair Popple wrote: On Tuesday, 9 February 2021 9:27:05 PM AEDT Daniel Vetter wrote: Recent changes to pin_user_pages() prevent the creation of pinned pages in ZONE_MOVABLE. This series allows pinned pages to be

Re: [PATCH 0/9] Add support for SVM atomics in Nouveau

2021-02-09 Thread Jason Gunthorpe
On Tue, Feb 09, 2021 at 02:39:51PM +0100, Daniel Vetter wrote: > Either way ZONE_DEVICE for not vram/device memory sounds wrong. Is > that really going on here? My read was this was doing non-coherent atomics on CPU memory. Atomics on GPU memory is just called migration to GPU memory, it

Re: [PATCH 0/9] Add support for SVM atomics in Nouveau

2021-02-09 Thread Daniel Vetter
On Tue, Feb 9, 2021 at 2:35 PM Jason Gunthorpe wrote: > > On Tue, Feb 09, 2021 at 11:57:28PM +1100, Alistair Popple wrote: > > On Tuesday, 9 February 2021 9:27:05 PM AEDT Daniel Vetter wrote: > > > > > > > > Recent changes to pin_user_pages() prevent the creation of pinned pages > > > > in > > >

Re: [PATCH 0/9] Add support for SVM atomics in Nouveau

2021-02-09 Thread Daniel Vetter
On Tue, Feb 9, 2021 at 1:57 PM Alistair Popple wrote: > > On Tuesday, 9 February 2021 9:27:05 PM AEDT Daniel Vetter wrote: > > > > > > Recent changes to pin_user_pages() prevent the creation of pinned pages in > > > ZONE_MOVABLE. This series allows pinned pages to be created in > ZONE_MOVABLE > >

Re: [PATCH 0/9] Add support for SVM atomics in Nouveau

2021-02-09 Thread Jason Gunthorpe
On Tue, Feb 09, 2021 at 11:57:28PM +1100, Alistair Popple wrote: > On Tuesday, 9 February 2021 9:27:05 PM AEDT Daniel Vetter wrote: > > > > > > Recent changes to pin_user_pages() prevent the creation of pinned pages in > > > ZONE_MOVABLE. This series allows pinned pages to be created in >

Re: [PATCH 0/9] Add support for SVM atomics in Nouveau

2021-02-09 Thread Alistair Popple
On Tuesday, 9 February 2021 9:27:05 PM AEDT Daniel Vetter wrote: > > > > Recent changes to pin_user_pages() prevent the creation of pinned pages in > > ZONE_MOVABLE. This series allows pinned pages to be created in ZONE_MOVABLE > > as attempts to migrate may fail which would be fatal to

Re: [PATCH 0/9] Add support for SVM atomics in Nouveau

2021-02-09 Thread Daniel Vetter
On Tue, Feb 09, 2021 at 12:07:13PM +1100, Alistair Popple wrote: > This series adds support to Nouveau for atomic memory operations on OpenCL > shared virtual memory (SVM). This is achieved using the atomic PTE bits on > the GPU to only permit atomic operations to system memory when a page is >

[PATCH 0/9] Add support for SVM atomics in Nouveau

2021-02-08 Thread Alistair Popple
This series adds support to Nouveau for atomic memory operations on OpenCL shared virtual memory (SVM). This is achieved using the atomic PTE bits on the GPU to only permit atomic operations to system memory when a page is not mapped in userspace on the CPU. This is implemented by adding a mode