Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-07 Thread Tomasz Figa
On Wed, Oct 7, 2020 at 4:23 PM Daniel Vetter wrote: > > On Wed, Oct 7, 2020 at 4:12 PM Tomasz Figa wrote: > > > > On Wed, Oct 7, 2020 at 4:09 PM Daniel Vetter wrote: > > > > > > On Wed, Oct 7, 2020 at 3:34 PM Tomasz Figa wrote: > > > > > > > > On Wed, Oct 7, 2020 at 3:06 PM Jason Gunthorpe

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-07 Thread Jason Gunthorpe
On Wed, Oct 07, 2020 at 04:11:59PM +0200, Tomasz Figa wrote: > We also need to bring back the vma_open() that somehow disappeared > around 4.2, as Marek found. No Jason

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-07 Thread Daniel Vetter
On Wed, Oct 7, 2020 at 4:12 PM Tomasz Figa wrote: > > On Wed, Oct 7, 2020 at 4:09 PM Daniel Vetter wrote: > > > > On Wed, Oct 7, 2020 at 3:34 PM Tomasz Figa wrote: > > > > > > On Wed, Oct 7, 2020 at 3:06 PM Jason Gunthorpe wrote: > > > > > > > > On Wed, Oct 07, 2020 at 02:58:33PM +0200, Daniel

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-07 Thread Tomasz Figa
On Wed, Oct 7, 2020 at 4:09 PM Daniel Vetter wrote: > > On Wed, Oct 7, 2020 at 3:34 PM Tomasz Figa wrote: > > > > On Wed, Oct 7, 2020 at 3:06 PM Jason Gunthorpe wrote: > > > > > > On Wed, Oct 07, 2020 at 02:58:33PM +0200, Daniel Vetter wrote: > > > > On Wed, Oct 7, 2020 at 2:48 PM Tomasz Figa

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-07 Thread Daniel Vetter
On Wed, Oct 7, 2020 at 3:34 PM Tomasz Figa wrote: > > On Wed, Oct 7, 2020 at 3:06 PM Jason Gunthorpe wrote: > > > > On Wed, Oct 07, 2020 at 02:58:33PM +0200, Daniel Vetter wrote: > > > On Wed, Oct 7, 2020 at 2:48 PM Tomasz Figa wrote: > > > > > > > > On Wed, Oct 7, 2020 at 2:44 PM Jason

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-07 Thread Jason Gunthorpe
On Wed, Oct 07, 2020 at 03:34:01PM +0200, Tomasz Figa wrote: > I think the userptr zero-copy hack should be able to go away indeed, > given that we now have CMA that allows having carveouts backed by > struct pages and having the memory represented as DMA-buf normally. This also needs to figure

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-07 Thread Tomasz Figa
On Wed, Oct 7, 2020 at 3:06 PM Jason Gunthorpe wrote: > > On Wed, Oct 07, 2020 at 02:58:33PM +0200, Daniel Vetter wrote: > > On Wed, Oct 7, 2020 at 2:48 PM Tomasz Figa wrote: > > > > > > On Wed, Oct 7, 2020 at 2:44 PM Jason Gunthorpe wrote: > > > > > > > > On Wed, Oct 07, 2020 at 02:33:56PM

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-07 Thread Jason Gunthorpe
On Wed, Oct 07, 2020 at 03:06:17PM +0200, Tomasz Figa wrote: > Note that vb2_vmalloc is only used for in-kernel CPU usage, e.g. the > contents being copied by the driver between vb2 buffers and some > hardware FIFO or other dedicated buffers. The memory does not go to > any hardware DMA. That is

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-07 Thread Tomasz Figa
On Sat, Oct 3, 2020 at 1:31 AM Jason Gunthorpe wrote: > > On Fri, Oct 02, 2020 at 08:16:48PM +0200, Daniel Vetter wrote: > > On Fri, Oct 2, 2020 at 8:06 PM Jason Gunthorpe wrote: > > > On Fri, Oct 02, 2020 at 07:53:03PM +0200, Daniel Vetter wrote: > > > > For $reasons I've stumbled over this

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-07 Thread Jason Gunthorpe
On Wed, Oct 07, 2020 at 02:58:33PM +0200, Daniel Vetter wrote: > On Wed, Oct 7, 2020 at 2:48 PM Tomasz Figa wrote: > > > > On Wed, Oct 7, 2020 at 2:44 PM Jason Gunthorpe wrote: > > > > > > On Wed, Oct 07, 2020 at 02:33:56PM +0200, Marek Szyprowski wrote: > > > > Well, it was in vb2_get_vma()

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-07 Thread Daniel Vetter
On Wed, Oct 7, 2020 at 2:48 PM Tomasz Figa wrote: > > On Wed, Oct 7, 2020 at 2:44 PM Jason Gunthorpe wrote: > > > > On Wed, Oct 07, 2020 at 02:33:56PM +0200, Marek Szyprowski wrote: > > > Well, it was in vb2_get_vma() function, but now I see that it has been > > > lost in fb639eb39154 and

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-07 Thread Tomasz Figa
On Wed, Oct 7, 2020 at 2:44 PM Jason Gunthorpe wrote: > > On Wed, Oct 07, 2020 at 02:33:56PM +0200, Marek Szyprowski wrote: > > Well, it was in vb2_get_vma() function, but now I see that it has been > > lost in fb639eb39154 and 6690c8c78c74 some time ago... > > There is no guarentee that holding

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-07 Thread Jason Gunthorpe
On Wed, Oct 07, 2020 at 02:33:56PM +0200, Marek Szyprowski wrote: > Well, it was in vb2_get_vma() function, but now I see that it has been > lost in fb639eb39154 and 6690c8c78c74 some time ago... There is no guarentee that holding a get on the file says anthing about the VMA. This needed to

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-07 Thread Marek Szyprowski
Hi Daniel, On 07.10.2020 14:01, Daniel Vetter wrote: > On Wed, Oct 7, 2020 at 12:47 PM Marek Szyprowski > wrote: >> On 03.10.2020 11:40, Daniel Vetter wrote: After he three places above should use pin_user_pages_fast(), then this whole broken API should be moved into videobuf2-memops.c

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-07 Thread Daniel Vetter
On Wed, Oct 7, 2020 at 12:47 PM Marek Szyprowski wrote: > > Hi Daniel, > > On 03.10.2020 11:40, Daniel Vetter wrote: > >> After he three places above should use pin_user_pages_fast(), then > >> this whole broken API should be moved into videobuf2-memops.c and a > >> big fat "THIS DOESN'T WORK"

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-07 Thread Marek Szyprowski
Hi Daniel, On 03.10.2020 11:40, Daniel Vetter wrote: >> After he three places above should use pin_user_pages_fast(), then >> this whole broken API should be moved into videobuf2-memops.c and a >> big fat "THIS DOESN'T WORK" stuck on it. >> >> videobuf2 should probably use P2P DMA buf for this

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-06 Thread Daniel Vetter
On Tue, Oct 6, 2020 at 2:26 PM Jason Gunthorpe wrote: > > On Tue, Oct 06, 2020 at 08:23:23AM +0200, Daniel Vetter wrote: > > On Tue, Oct 6, 2020 at 1:41 AM Jason Gunthorpe wrote: > > > > > > On Tue, Oct 06, 2020 at 12:43:31AM +0200, Daniel Vetter wrote: > > > > > > > > iow I think I can outright

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-06 Thread Jason Gunthorpe
On Tue, Oct 06, 2020 at 08:23:23AM +0200, Daniel Vetter wrote: > On Tue, Oct 6, 2020 at 1:41 AM Jason Gunthorpe wrote: > > > > On Tue, Oct 06, 2020 at 12:43:31AM +0200, Daniel Vetter wrote: > > > > > > iow I think I can outright delete the frame vector stuff. > > > > > > Ok this doesn't work,

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-06 Thread Jason Gunthorpe
On Mon, Oct 05, 2020 at 08:36:00PM -0700, Andrew Morton wrote: > On Mon, 5 Oct 2020 14:47:47 -0300 Jason Gunthorpe wrote: > > > Andrew please let me know if you need a resend > > Andrew is rather confused. > > Can we please identify the minimal patch(es) which are needed for 5.9 > and -stable?

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-06 Thread Daniel Vetter
On Mon, Oct 5, 2020 at 7:58 PM Jason Gunthorpe wrote: > > On Mon, Oct 05, 2020 at 07:53:08PM +0200, Jan Kara wrote: > > On Mon 05-10-20 14:38:54, Jason Gunthorpe wrote: > > > When get_vaddr_frames() does its hacky follow_pfn() loop it should never > > > be allowed to extract a struct page from a

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-06 Thread Daniel Vetter
On Tue, Oct 6, 2020 at 1:41 AM Jason Gunthorpe wrote: > > On Tue, Oct 06, 2020 at 12:43:31AM +0200, Daniel Vetter wrote: > > > > iow I think I can outright delete the frame vector stuff. > > > > Ok this doesn't work, because dma_mmap always uses a remap_pfn_range, > > which is a VM_IO | VM_PFNMAP

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-05 Thread Andrew Morton
On Mon, 5 Oct 2020 14:47:47 -0300 Jason Gunthorpe wrote: > Andrew please let me know if you need a resend Andrew is rather confused. Can we please identify the minimal patch(es) which are needed for 5.9 and -stable?

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-05 Thread Jason Gunthorpe
On Tue, Oct 06, 2020 at 12:43:31AM +0200, Daniel Vetter wrote: > > iow I think I can outright delete the frame vector stuff. > > Ok this doesn't work, because dma_mmap always uses a remap_pfn_range, > which is a VM_IO | VM_PFNMAP vma and so even if it's cma backed and > not a carveout, we can't

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-05 Thread Daniel Vetter
On Mon, Oct 5, 2020 at 8:54 PM Daniel Vetter wrote: > > On Mon, Oct 5, 2020 at 8:37 PM Jason Gunthorpe wrote: > > > > On Mon, Oct 05, 2020 at 08:16:33PM +0200, Daniel Vetter wrote: > > > > > > kvm is some similar hack added for P2P DMA, see commit > > > >

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-05 Thread Daniel Vetter
On Mon, Oct 5, 2020 at 8:37 PM Jason Gunthorpe wrote: > > On Mon, Oct 05, 2020 at 08:16:33PM +0200, Daniel Vetter wrote: > > > > kvm is some similar hack added for P2P DMA, see commit > > > add6a0cd1c5ba51b201e1361b05a5df817083618. It might be protected by > > > notifiers.. > > > > Yeah my

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-05 Thread Jason Gunthorpe
On Mon, Oct 05, 2020 at 08:16:33PM +0200, Daniel Vetter wrote: > > kvm is some similar hack added for P2P DMA, see commit > > add6a0cd1c5ba51b201e1361b05a5df817083618. It might be protected by > > notifiers.. > > Yeah my thinking is that kvm (and I think also vfio, also seems to > have mmu

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-05 Thread Daniel Vetter
On Mon, Oct 5, 2020 at 7:58 PM Jason Gunthorpe wrote: > > On Mon, Oct 05, 2020 at 07:53:08PM +0200, Jan Kara wrote: > > On Mon 05-10-20 14:38:54, Jason Gunthorpe wrote: > > > When get_vaddr_frames() does its hacky follow_pfn() loop it should never > > > be allowed to extract a struct page from a

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-05 Thread Daniel Vetter
On Mon, Oct 5, 2020 at 7:28 PM Jason Gunthorpe wrote: > > On Sun, Oct 04, 2020 at 06:09:29PM +0200, Daniel Vetter wrote: > > On Sun, Oct 4, 2020 at 2:51 PM Jason Gunthorpe wrote: > > > > > > On Sat, Oct 03, 2020 at 11:40:22AM +0200, Daniel Vetter wrote: > > > > > > > > That leaves the only

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-05 Thread Jason Gunthorpe
On Mon, Oct 05, 2020 at 07:53:08PM +0200, Jan Kara wrote: > On Mon 05-10-20 14:38:54, Jason Gunthorpe wrote: > > When get_vaddr_frames() does its hacky follow_pfn() loop it should never > > be allowed to extract a struct page from a normal VMA. This could allow a > > serious use-after-free problem

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-05 Thread Jason Gunthorpe
On Mon, Oct 05, 2020 at 02:38:54PM -0300, Jason Gunthorpe wrote: > When get_vaddr_frames() does its hacky follow_pfn() loop it should never > be allowed to extract a struct page from a normal VMA. This could allow a > serious use-after-free problem on any kernel memory. > > Restrict this to only

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-05 Thread Jan Kara
On Mon 05-10-20 14:38:54, Jason Gunthorpe wrote: > When get_vaddr_frames() does its hacky follow_pfn() loop it should never > be allowed to extract a struct page from a normal VMA. This could allow a > serious use-after-free problem on any kernel memory. > > Restrict this to only work on VMA's

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-05 Thread Jason Gunthorpe
When get_vaddr_frames() does its hacky follow_pfn() loop it should never be allowed to extract a struct page from a normal VMA. This could allow a serious use-after-free problem on any kernel memory. Restrict this to only work on VMA's with one of VM_IO | VM_PFNMAP set. This limits the

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-05 Thread Jason Gunthorpe
On Sun, Oct 04, 2020 at 01:20:31PM +0200, Daniel Vetter wrote: > Yeah I think that works. I tried understanding gup.c code a bit more, > and it looks like FOLL_LONGTERM only works for the pup_fast variant > right now? All others seem to have this comment that it's somehow > incompatible with

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-05 Thread Jason Gunthorpe
On Sun, Oct 04, 2020 at 06:09:29PM +0200, Daniel Vetter wrote: > On Sun, Oct 4, 2020 at 2:51 PM Jason Gunthorpe wrote: > > > > On Sat, Oct 03, 2020 at 11:40:22AM +0200, Daniel Vetter wrote: > > > > > > That leaves the only interesting places as vb2_dc_get_userptr() and > > > >

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-05 Thread Jan Kara
Hi! On Fri 02-10-20 15:06:03, Jason Gunthorpe wrote: > This get_vaddr_frames() thing looks impossible to use properly. How on > earth does a driver guarentee > > "If @start belongs to VM_IO | VM_PFNMAP vma, we don't touch page > structures and the caller must make sure pfns aren't reused for >

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-04 Thread Daniel Vetter
On Sun, Oct 4, 2020 at 2:51 PM Jason Gunthorpe wrote: > > On Sat, Oct 03, 2020 at 11:40:22AM +0200, Daniel Vetter wrote: > > > > That leaves the only interesting places as vb2_dc_get_userptr() and > > > vb2_vmalloc_get_userptr() which both completely fail to follow the > > > REQUIRED behavior in

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-04 Thread Jason Gunthorpe
On Sat, Oct 03, 2020 at 11:40:22AM +0200, Daniel Vetter wrote: > > That leaves the only interesting places as vb2_dc_get_userptr() and > > vb2_vmalloc_get_userptr() which both completely fail to follow the > > REQUIRED behavior in the function's comment about checking PTEs. It > > just DMA maps

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-04 Thread Daniel Vetter
On Sun, Oct 4, 2020 at 1:24 AM Jason Gunthorpe wrote: > On Sat, Oct 03, 2020 at 03:52:32PM -0700, John Hubbard wrote: > > On 10/3/20 2:45 AM, Daniel Vetter wrote: > > > On Sat, Oct 3, 2020 at 12:39 AM John Hubbard wrote: > > > > > > > > On 10/2/20 10:53 AM, Daniel Vetter wrote: > > > > > For

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-03 Thread Jason Gunthorpe
On Sat, Oct 03, 2020 at 03:52:32PM -0700, John Hubbard wrote: > On 10/3/20 2:45 AM, Daniel Vetter wrote: > > On Sat, Oct 3, 2020 at 12:39 AM John Hubbard wrote: > > > > > > On 10/2/20 10:53 AM, Daniel Vetter wrote: > > > > For $reasons I've stumbled over this code and I'm not sure the change > >

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-03 Thread John Hubbard
On 10/3/20 2:45 AM, Daniel Vetter wrote: On Sat, Oct 3, 2020 at 12:39 AM John Hubbard wrote: On 10/2/20 10:53 AM, Daniel Vetter wrote: For $reasons I've stumbled over this code and I'm not sure the change to the new gup functions in 55a650c35fea ("mm/gup: frame_vector: convert

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-03 Thread Daniel Vetter
On Sat, Oct 3, 2020 at 12:39 AM John Hubbard wrote: > > On 10/2/20 10:53 AM, Daniel Vetter wrote: > > For $reasons I've stumbled over this code and I'm not sure the change > > to the new gup functions in 55a650c35fea ("mm/gup: frame_vector: > > convert get_user_pages() --> pin_user_pages()") was

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-03 Thread Daniel Vetter
On Sat, Oct 3, 2020 at 1:31 AM Jason Gunthorpe wrote: > > On Fri, Oct 02, 2020 at 08:16:48PM +0200, Daniel Vetter wrote: > > On Fri, Oct 2, 2020 at 8:06 PM Jason Gunthorpe wrote: > > > On Fri, Oct 02, 2020 at 07:53:03PM +0200, Daniel Vetter wrote: > > > > For $reasons I've stumbled over this

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-03 Thread Oded Gabbay
On Sat, Oct 3, 2020 at 2:31 AM Jason Gunthorpe wrote: > > On Fri, Oct 02, 2020 at 08:16:48PM +0200, Daniel Vetter wrote: > > On Fri, Oct 2, 2020 at 8:06 PM Jason Gunthorpe wrote: > > > On Fri, Oct 02, 2020 at 07:53:03PM +0200, Daniel Vetter wrote: > > > > For $reasons I've stumbled over this

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-02 Thread Jason Gunthorpe
On Fri, Oct 02, 2020 at 08:16:48PM +0200, Daniel Vetter wrote: > On Fri, Oct 2, 2020 at 8:06 PM Jason Gunthorpe wrote: > > On Fri, Oct 02, 2020 at 07:53:03PM +0200, Daniel Vetter wrote: > > > For $reasons I've stumbled over this code and I'm not sure the change > > > to the new gup functions in

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-02 Thread John Hubbard
On 10/2/20 10:53 AM, Daniel Vetter wrote: For $reasons I've stumbled over this code and I'm not sure the change to the new gup functions in 55a650c35fea ("mm/gup: frame_vector: convert get_user_pages() --> pin_user_pages()") was entirely correct. This here is used for long term buffers (not

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-02 Thread Daniel Vetter
On Fri, Oct 2, 2020 at 8:06 PM Jason Gunthorpe wrote: > On Fri, Oct 02, 2020 at 07:53:03PM +0200, Daniel Vetter wrote: > > For $reasons I've stumbled over this code and I'm not sure the change > > to the new gup functions in 55a650c35fea ("mm/gup: frame_vector: > > convert get_user_pages() -->

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-02 Thread Jason Gunthorpe
On Fri, Oct 02, 2020 at 07:53:03PM +0200, Daniel Vetter wrote: > For $reasons I've stumbled over this code and I'm not sure the change > to the new gup functions in 55a650c35fea ("mm/gup: frame_vector: > convert get_user_pages() --> pin_user_pages()") was entirely correct. > > This here is used

[PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-02 Thread Daniel Vetter
For $reasons I've stumbled over this code and I'm not sure the change to the new gup functions in 55a650c35fea ("mm/gup: frame_vector: convert get_user_pages() --> pin_user_pages()") was entirely correct. This here is used for long term buffers (not just quick I/O) like RDMA, and John notes this