Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2019-01-15 Thread John Hubbard
On 1/15/19 12:34 AM, Jan Kara wrote: > On Mon 14-01-19 11:09:20, John Hubbard wrote: >> On 1/14/19 9:21 AM, Jerome Glisse wrote: >>>> [...] > >> For example, the following already survives a basic boot to graphics mode. >> It requires a bunch of callsite c

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2019-01-14 Thread John Hubbard
On 1/14/19 9:21 AM, Jerome Glisse wrote: > On Mon, Jan 14, 2019 at 03:54:47PM +0100, Jan Kara wrote: >> On Fri 11-01-19 19:06:08, John Hubbard wrote: >>> On 1/11/19 6:46 PM, Jerome Glisse wrote: >>>> On Fri, Jan 11, 2019 at 06:38:44PM -0800, John Hubbard wrote: >

[PATCH 1/1] phy: fix build breakage: add PHY_MODE_SATA

2019-01-12 Thread john . hubbard
From: John Hubbard Commit 49e54187ae0b ("ata: libahci_platform: comply to PHY framework") uses the PHY_MODE_SATA, but that enum had not yet been added. This caused a build failure for me, with today's linux.git. Also, there is a potentially conflicting (mis-named) PHY_MODE_SA

[PATCH 0/1] PHY_MODE_SATA build fix

2019-01-12 Thread john . hubbard
From: John Hubbard Hi, Say, I just ran into this build breakage on today's linux.git, and after checking the email threads, I do realize that PHY_MODE_SATA is about to be added as part of [1]. However, I also noticed that it was not identified as a build fix, nor did anyone notice the potential

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2019-01-12 Thread John Hubbard
ases that will highlight whether page lock is a noticeable problem here. Or, the other thing that could happen is someone will jump in here and NAK anything involving the page lock, based on long experience, and we'll just go straight to your scheme anyway. I'm sorta expecting that any minute now. :) thanks, -- John Hubbard NVIDIA

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2019-01-11 Thread John Hubbard
On 1/11/19 6:46 PM, Jerome Glisse wrote: > On Fri, Jan 11, 2019 at 06:38:44PM -0800, John Hubbard wrote: >> On 1/11/19 6:02 PM, Jerome Glisse wrote: >>> On Fri, Jan 11, 2019 at 05:04:05PM -0800, John Hubbard wrote: >>>> On 1/11/19 8:51 AM, Jerome Glisse wrote: >&

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2019-01-11 Thread John Hubbard
On 1/11/19 6:02 PM, Jerome Glisse wrote: > On Fri, Jan 11, 2019 at 05:04:05PM -0800, John Hubbard wrote: >> On 1/11/19 8:51 AM, Jerome Glisse wrote: >>> On Thu, Jan 10, 2019 at 06:59:31PM -0800, John Hubbard wrote: >>>> On 1/3/19 6:44 AM, Jerome Glisse wrote: >&

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2019-01-11 Thread John Hubbard
On 1/11/19 8:51 AM, Jerome Glisse wrote: > On Thu, Jan 10, 2019 at 06:59:31PM -0800, John Hubbard wrote: >> On 1/3/19 6:44 AM, Jerome Glisse wrote: >>> On Thu, Jan 03, 2019 at 10:26:54AM +0100, Jan Kara wrote: >>>> On Wed 02-01-19 20:55:33, Jerome Glisse wrote: >&

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2019-01-10 Thread John Hubbard
t;>>>> On Tue, Dec 18, 2018 at 03:29:34PM -0800, John Hubbard wrote: >>>>>> OK, so let's take another look at Jerome's _mapcount idea all by itself >>>>>> (using >>>>>> *only* the tracking pinned pages aspect), given that it is the

Re: [PATCH] Initialise mmu_notifier_range correctly

2019-01-02 Thread John Hubbard
gt;> ptep = pte_offset_map_lock(mm, pmd, address, ptlp); > Looks correct to me, as well. Having the range struct declared in separate places from the mmu_notifier_range_init() calls is not great. But I'm not sure I see a way to make it significantly cleaner, given that __follow_pte_pmd uses the range pointer as a way to decide to issue the mmn calls. thanks, -- John Hubbard NVIDIA

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2019-01-02 Thread John Hubbard
On 1/2/19 5:55 PM, Jerome Glisse wrote: > On Wed, Dec 19, 2018 at 12:08:56PM +0100, Jan Kara wrote: >> On Tue 18-12-18 21:07:24, Jerome Glisse wrote: >>> On Tue, Dec 18, 2018 at 03:29:34PM -0800, John Hubbard wrote: >>>> OK, so let's take another look at Jerome'

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-20 Thread John Hubbard
On 12/19/18 3:08 AM, Jan Kara wrote: > On Tue 18-12-18 21:07:24, Jerome Glisse wrote: >> On Tue, Dec 18, 2018 at 03:29:34PM -0800, John Hubbard wrote: >>> OK, so let's take another look at Jerome's _mapcount idea all by itself >>> (using >>> *only* the

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-18 Thread John Hubbard
n that it is the lightest weight solution for that. So as I understand it, this would use page->_mapcount to store both the real mapcount, and the dma pinned count (simply added together), but only do so for file-backed (non-anonymous) pages: __get_user_pages() { ... get_page(page); if (!PageAnon) atomic_inc(page->_mapcount); ... } put_user_page(struct page *page) { ... if (!PageAnon) atomic_dec(>_mapcount); put_page(page); ... } ...and then in the various consumers of the DMA pinned count, we use page_mapped(page) to see if any mapcount remains, and if so, we treat it as DMA pinned. Is that what you had in mind? -- thanks, John Hubbard NVIDIA

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-14 Thread John Hubbard
rs really are complex) really seems worth the extra work, so that's a big benefit. Next steps: I want to go try this dynamic_page approach out right away. If there are pieces such as page_to_pfn and related, that are already in progress, I'd definitely like to work on top of that. Also, any up front advice or pitfalls to avoid is always welcome, of course. :) thanks, -- John Hubbard NVIDIA

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-13 Thread John Hubbard
On 12/13/18 9:21 PM, Dan Williams wrote: > On Thu, Dec 13, 2018 at 7:53 PM John Hubbard wrote: >> >> On 12/12/18 4:51 PM, Dave Chinner wrote: >>> On Wed, Dec 12, 2018 at 04:59:31PM -0500, Jerome Glisse wrote: >>>> On Thu, Dec 13, 2018 at 08:46:41AM +1100, Dave

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-13 Thread John Hubbard
an't call ->page_mkwrite() from >>> put_user_page(), so I don't think this is workable at all. >> >> Hu why ? i can not think of any reason whike you could not. User of > > It's not a fault path, you can't safely lock pages, you can't take > fault-path only locks in the IO path (mmap_sem inversion problems), > etc. > Yes, I looked closer at ->page_mkwrite (ext4_page_mkwrite, for example), and it's clearly doing lock_page(), so it does seem like this particular detail (calling page_mkwrite from put_user_page) is dead. > /me has a nagging feeling this was all explained in a previous > discussions of this patchset... > Yes, lots of related discussion definitely happened already, for example this October thread covered page_mkwrite and interactions with gup: https://lore.kernel.org/r/20181001061127.GQ31060@dastard ...but so far, this is the first time I recall seeing a proposal to call page_mkwrite from put_user_page. thanks, -- John Hubbard NVIDIA

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-12 Thread John Hubbard
t can replay page faults, in many cases. I think as long as we specify that the acceptable consequence of doing, say, umount on a filesystem that has active DMA happening is that the associated processes get killed, then we're going to be OK. What would worry me is if there was an expectation that processes could continue working properly after such a scenario. thanks, -- John Hubbard NVIDIA

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-12 Thread John Hubbard
On 12/12/18 2:14 PM, Jerome Glisse wrote: > On Wed, Dec 12, 2018 at 02:11:58PM -0800, John Hubbard wrote: >> On 12/12/18 2:04 PM, Jerome Glisse wrote: >>> On Wed, Dec 12, 2018 at 01:56:00PM -0800, John Hubbard wrote: >>>> On 12/12/18 1:30 PM, Jerome Glisse wrote: &g

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-12 Thread John Hubbard
On 12/12/18 2:04 PM, Jerome Glisse wrote: > On Wed, Dec 12, 2018 at 01:56:00PM -0800, John Hubbard wrote: >> On 12/12/18 1:30 PM, Jerome Glisse wrote: >>> On Wed, Dec 12, 2018 at 08:27:35AM -0800, Dan Williams wrote: >>>> On Wed, Dec 12, 2018 at 7:03 AM Jerome Gli

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-12 Thread John Hubbard
about it then trying to make it pass under the radar. > > This will put the burden on broken user and allow you to properly > recycle your DAX page. > > Think of it as revoke through mmu notifier. > > So patchset would be: > enum mmu_notifier_event { > + MMU_NOTI

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-12 Thread John Hubbard
On 12/7/18 9:18 PM, Matthew Wilcox wrote: > On Fri, Dec 07, 2018 at 04:52:42PM -0800, John Hubbard wrote: >> I see. OK, HMM has done an efficient job of mopping up unused fields, and >> now we are >> completely out of space. At this point, after thinking about it carefull

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-12 Thread John Hubbard
nned dax pages, see >>> dax_layout_busy_page(). As much as possible I want to eliminate the >>> concept of "dax pages" as a special case that gets sprinkled >>> throughout the mm. >>> >>>> For [O1] and [O2] i believe a solution with mapcount

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-07 Thread John Hubbard
On 12/7/18 11:16 AM, Jerome Glisse wrote: > On Thu, Dec 06, 2018 at 06:45:49PM -0800, John Hubbard wrote: >> On 12/4/18 5:57 PM, John Hubbard wrote: >>> On 12/4/18 5:44 PM, Jerome Glisse wrote: >>>> On Tue, Dec 04, 2018 at 05:15:19PM -0800, Matthew Wilcox wrote: >&

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-07 Thread John Hubbard
On 12/7/18 11:16 AM, Jerome Glisse wrote: > On Thu, Dec 06, 2018 at 06:45:49PM -0800, John Hubbard wrote: >> On 12/4/18 5:57 PM, John Hubbard wrote: >>> On 12/4/18 5:44 PM, Jerome Glisse wrote: >>>> On Tue, Dec 04, 2018 at 05:15:19PM -0800, Matthew Wilcox wrote: >&

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-06 Thread John Hubbard
On 12/4/18 5:57 PM, John Hubbard wrote: > On 12/4/18 5:44 PM, Jerome Glisse wrote: >> On Tue, Dec 04, 2018 at 05:15:19PM -0800, Matthew Wilcox wrote: >>> On Tue, Dec 04, 2018 at 04:58:01PM -0800, John Hubbard wrote: >>>> On 12/4/18 3:03 PM, Dan Williams wrot

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-06 Thread John Hubbard
On 12/4/18 5:57 PM, John Hubbard wrote: > On 12/4/18 5:44 PM, Jerome Glisse wrote: >> On Tue, Dec 04, 2018 at 05:15:19PM -0800, Matthew Wilcox wrote: >>> On Tue, Dec 04, 2018 at 04:58:01PM -0800, John Hubbard wrote: >>>> On 12/4/18 3:03 PM, Dan Williams wrot

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-04 Thread John Hubbard
On 12/4/18 5:44 PM, Jerome Glisse wrote: > On Tue, Dec 04, 2018 at 05:15:19PM -0800, Matthew Wilcox wrote: >> On Tue, Dec 04, 2018 at 04:58:01PM -0800, John Hubbard wrote: >>> On 12/4/18 3:03 PM, Dan Williams wrote: >>>> Except the LRU fields are already in us

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-04 Thread John Hubbard
On 12/4/18 5:44 PM, Jerome Glisse wrote: > On Tue, Dec 04, 2018 at 05:15:19PM -0800, Matthew Wilcox wrote: >> On Tue, Dec 04, 2018 at 04:58:01PM -0800, John Hubbard wrote: >>> On 12/4/18 3:03 PM, Dan Williams wrote: >>>> Except the LRU fields are already in us

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-04 Thread John Hubbard
On 12/3/18 11:53 PM, Mike Rapoport wrote: > Hi John, > > Thanks for having documentation as a part of the patch. Some kernel-doc > nits below. > > On Mon, Dec 03, 2018 at 04:17:19PM -0800, john.hubb...@gmail.com wrote: >> From: John Hubbard >> >> Introduces

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-04 Thread John Hubbard
On 12/3/18 11:53 PM, Mike Rapoport wrote: > Hi John, > > Thanks for having documentation as a part of the patch. Some kernel-doc > nits below. > > On Mon, Dec 03, 2018 at 04:17:19PM -0800, john.hubb...@gmail.com wrote: >> From: John Hubbard >> >> Introduces

Re: [PATCH 0/2] put_user_page*(): start converting the call sites

2018-12-04 Thread John Hubbard
hich I should have added in this cover letter. Here's a start: https://lore.kernel.org/r/20181110085041.10071-1-jhubb...@nvidia.com ...and it looks like this small patch series is not going to work out--I'm going to have to fall back to another RFC spin. So I'll be sure to include you and everyone on that. Hope that helps. thanks, -- John Hubbard NVIDIA

Re: [PATCH 0/2] put_user_page*(): start converting the call sites

2018-12-04 Thread John Hubbard
hich I should have added in this cover letter. Here's a start: https://lore.kernel.org/r/20181110085041.10071-1-jhubb...@nvidia.com ...and it looks like this small patch series is not going to work out--I'm going to have to fall back to another RFC spin. So I'll be sure to include you and everyone on that. Hope that helps. thanks, -- John Hubbard NVIDIA

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-04 Thread John Hubbard
On 12/4/18 4:40 PM, Dan Williams wrote: > On Tue, Dec 4, 2018 at 4:37 PM Jerome Glisse wrote: >> >> On Tue, Dec 04, 2018 at 03:03:02PM -0800, Dan Williams wrote: >>> On Tue, Dec 4, 2018 at 1:56 PM John Hubbard wrote: >>>> >>>> On 12/4/18 12:28 P

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-04 Thread John Hubbard
On 12/4/18 4:40 PM, Dan Williams wrote: > On Tue, Dec 4, 2018 at 4:37 PM Jerome Glisse wrote: >> >> On Tue, Dec 04, 2018 at 03:03:02PM -0800, Dan Williams wrote: >>> On Tue, Dec 4, 2018 at 1:56 PM John Hubbard wrote: >>>> >>>> On 12/4/18 12:28 P

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-04 Thread John Hubbard
On 12/4/18 3:03 PM, Dan Williams wrote: > On Tue, Dec 4, 2018 at 1:56 PM John Hubbard wrote: >> >> On 12/4/18 12:28 PM, Dan Williams wrote: >>> On Mon, Dec 3, 2018 at 4:17 PM wrote: >>>> >>>> From: John Hubbard >>>>

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-04 Thread John Hubbard
On 12/4/18 3:03 PM, Dan Williams wrote: > On Tue, Dec 4, 2018 at 1:56 PM John Hubbard wrote: >> >> On 12/4/18 12:28 PM, Dan Williams wrote: >>> On Mon, Dec 3, 2018 at 4:17 PM wrote: >>>> >>>> From: John Hubbard >>>>

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-04 Thread John Hubbard
On 12/4/18 12:28 PM, Dan Williams wrote: > On Mon, Dec 3, 2018 at 4:17 PM wrote: >> >> From: John Hubbard >> >> Introduces put_user_page(), which simply calls put_page(). >> This provides a way to update all get_user_pages*() callers, >> so that they ca

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-04 Thread John Hubbard
On 12/4/18 12:28 PM, Dan Williams wrote: > On Mon, Dec 3, 2018 at 4:17 PM wrote: >> >> From: John Hubbard >> >> Introduces put_user_page(), which simply calls put_page(). >> This provides a way to update all get_user_pages*() callers, >> so that they ca

[PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-03 Thread john . hubbard
From: John Hubbard Introduces put_user_page(), which simply calls put_page(). This provides a way to update all get_user_pages*() callers, so that they call put_user_page(), instead of put_page(). Also introduces put_user_pages(), and a few dirty/locked variations, as a replacement

[PATCH 2/2] infiniband/mm: convert put_page() to put_user_page*()

2018-12-03 Thread john . hubbard
From: John Hubbard For infiniband code that retains pages via get_user_pages*(), release those pages via the new put_user_page(), or put_user_pages*(), instead of put_page() This is a tiny part of the second step of fixing the problem described in [1]. The steps are: 1) Provide put_user_page

[PATCH 0/2] put_user_page*(): start converting the call sites

2018-12-03 Thread john . hubbard
From: John Hubbard Hi, Summary: I'd like these two patches to go into the next convenient cycle. I *think* that means 4.21. Details At the Linux Plumbers Conference, we talked about this approach [1], and the primary lingering concern was over performance. Tom Talpey helped me through a much

[PATCH 0/2] put_user_page*(): start converting the call sites

2018-12-03 Thread john . hubbard
From: John Hubbard Hi, Summary: I'd like these two patches to go into the next convenient cycle. I *think* that means 4.21. Details At the Linux Plumbers Conference, we talked about this approach [1], and the primary lingering concern was over performance. Tom Talpey helped me through a much

[PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-03 Thread john . hubbard
From: John Hubbard Introduces put_user_page(), which simply calls put_page(). This provides a way to update all get_user_pages*() callers, so that they call put_user_page(), instead of put_page(). Also introduces put_user_pages(), and a few dirty/locked variations, as a replacement

[PATCH 2/2] infiniband/mm: convert put_page() to put_user_page*()

2018-12-03 Thread john . hubbard
From: John Hubbard For infiniband code that retains pages via get_user_pages*(), release those pages via the new put_user_page(), or put_user_pages*(), instead of put_page() This is a tiny part of the second step of fixing the problem described in [1]. The steps are: 1) Provide put_user_page

Re: [PATCH v2 0/6] RFC: gup+dma: tracking dma-pinned pages

2018-11-29 Thread John Hubbard
On 11/29/18 6:30 PM, Tom Talpey wrote: > On 11/29/2018 9:21 PM, John Hubbard wrote: >> On 11/29/18 6:18 PM, Tom Talpey wrote: >>> On 11/29/2018 8:39 PM, John Hubbard wrote: >>>> On 11/28/18 5:59 AM, Tom Talpey wrote: >>>>> On 11/27/2018 9:52 PM, John H

Re: [PATCH v2 0/6] RFC: gup+dma: tracking dma-pinned pages

2018-11-29 Thread John Hubbard
On 11/29/18 6:30 PM, Tom Talpey wrote: > On 11/29/2018 9:21 PM, John Hubbard wrote: >> On 11/29/18 6:18 PM, Tom Talpey wrote: >>> On 11/29/2018 8:39 PM, John Hubbard wrote: >>>> On 11/28/18 5:59 AM, Tom Talpey wrote: >>>>> On 11/27/2018 9:52 PM, John H

Re: [PATCH v2 0/6] RFC: gup+dma: tracking dma-pinned pages

2018-11-29 Thread John Hubbard
On 11/29/18 6:18 PM, Tom Talpey wrote: > On 11/29/2018 8:39 PM, John Hubbard wrote: >> On 11/28/18 5:59 AM, Tom Talpey wrote: >>> On 11/27/2018 9:52 PM, John Hubbard wrote: >>>> On 11/27/18 5:21 PM, Tom Talpey wrote: >>>>> On 11/21/2018 5:06 PM, John H

Re: [PATCH v2 0/6] RFC: gup+dma: tracking dma-pinned pages

2018-11-29 Thread John Hubbard
On 11/29/18 6:18 PM, Tom Talpey wrote: > On 11/29/2018 8:39 PM, John Hubbard wrote: >> On 11/28/18 5:59 AM, Tom Talpey wrote: >>> On 11/27/2018 9:52 PM, John Hubbard wrote: >>>> On 11/27/18 5:21 PM, Tom Talpey wrote: >>>>> On 11/21/2018 5:06 PM, John H

Re: [PATCH v2 0/6] RFC: gup+dma: tracking dma-pinned pages

2018-11-29 Thread John Hubbard
On 11/28/18 5:59 AM, Tom Talpey wrote: > On 11/27/2018 9:52 PM, John Hubbard wrote: >> On 11/27/18 5:21 PM, Tom Talpey wrote: >>> On 11/21/2018 5:06 PM, John Hubbard wrote: >>>> On 11/21/18 8:49 AM, Tom Talpey wrote: >>>>> On 11/21/2018 1:09 AM, John Hu

Re: [PATCH v2 0/6] RFC: gup+dma: tracking dma-pinned pages

2018-11-29 Thread John Hubbard
On 11/28/18 5:59 AM, Tom Talpey wrote: > On 11/27/2018 9:52 PM, John Hubbard wrote: >> On 11/27/18 5:21 PM, Tom Talpey wrote: >>> On 11/21/2018 5:06 PM, John Hubbard wrote: >>>> On 11/21/18 8:49 AM, Tom Talpey wrote: >>>>> On 11/21/2018 1:09 AM, John Hu

Re: [PATCH v2 0/6] RFC: gup+dma: tracking dma-pinned pages

2018-11-27 Thread John Hubbard
On 11/27/18 5:21 PM, Tom Talpey wrote: > On 11/21/2018 5:06 PM, John Hubbard wrote: >> On 11/21/18 8:49 AM, Tom Talpey wrote: >>> On 11/21/2018 1:09 AM, John Hubbard wrote: >>>> On 11/19/18 10:57 AM, Tom Talpey wrote: [...] >>> >>> What I'd real

Re: [PATCH v2 0/6] RFC: gup+dma: tracking dma-pinned pages

2018-11-27 Thread John Hubbard
On 11/27/18 5:21 PM, Tom Talpey wrote: > On 11/21/2018 5:06 PM, John Hubbard wrote: >> On 11/21/18 8:49 AM, Tom Talpey wrote: >>> On 11/21/2018 1:09 AM, John Hubbard wrote: >>>> On 11/19/18 10:57 AM, Tom Talpey wrote: [...] >>> >>> What I'd real

Re: [PATCH] mm/gup: finish consolidating error handling

2018-11-21 Thread John Hubbard
his call", rather than the generic case of crossing a vma boundary. (I think there's a fine point that I must be overlooking.) But it's still a valid case, either way. -- thanks, John Hubbard NVIDIA

Re: [PATCH] mm/gup: finish consolidating error handling

2018-11-21 Thread John Hubbard
his call", rather than the generic case of crossing a vma boundary. (I think there's a fine point that I must be overlooking.) But it's still a valid case, either way. -- thanks, John Hubbard NVIDIA

Re: [PATCH v2 0/6] RFC: gup+dma: tracking dma-pinned pages

2018-11-21 Thread John Hubbard
On 11/21/18 8:49 AM, Tom Talpey wrote: > On 11/21/2018 1:09 AM, John Hubbard wrote: >> On 11/19/18 10:57 AM, Tom Talpey wrote: >>> ~14000 4KB read IOPS is really, really low for an NVMe disk. >> >> Yes, but Jan Kara's original config file for fio is *intended* to h

Re: [PATCH v2 0/6] RFC: gup+dma: tracking dma-pinned pages

2018-11-21 Thread John Hubbard
On 11/21/18 8:49 AM, Tom Talpey wrote: > On 11/21/2018 1:09 AM, John Hubbard wrote: >> On 11/19/18 10:57 AM, Tom Talpey wrote: >>> ~14000 4KB read IOPS is really, really low for an NVMe disk. >> >> Yes, but Jan Kara's original config file for fio is *intended* to h

[PATCH] mm/gup: finish consolidating error handling

2018-11-21 Thread john . hubbard
From: John Hubbard Commit df06b37ffe5a4 ("mm/gup: cache dev_pagemap while pinning pages") attempted to operate on each page that get_user_pages had retrieved. In order to do that, it created a common exit point from the routine. However, one case was missed, which this patch fixes

[PATCH] mm/gup: finish consolidating error handling

2018-11-21 Thread john . hubbard
From: John Hubbard Commit df06b37ffe5a4 ("mm/gup: cache dev_pagemap while pinning pages") attempted to operate on each page that get_user_pages had retrieved. In order to do that, it created a common exit point from the routine. However, one case was missed, which this patch fixes

[PATCH 0/1] mm/gup: finish consolidating error handling

2018-11-21 Thread john . hubbard
From: John Hubbard Hi, Keith Busch and Dan Williams noticed that this patch (which was part of my RFC[1] for the get_user_pages + DMA fix) also fixes a bug. Accordingly, I'm adjusting the changelog and posting this as it's own patch. [1] https://lkml.kernel.org/r/20181110085041.10071-1-jhubb

[PATCH 0/1] mm/gup: finish consolidating error handling

2018-11-21 Thread john . hubbard
From: John Hubbard Hi, Keith Busch and Dan Williams noticed that this patch (which was part of my RFC[1] for the get_user_pages + DMA fix) also fixes a bug. Accordingly, I'm adjusting the changelog and posting this as it's own patch. [1] https://lkml.kernel.org/r/20181110085041.10071-1-jhubb

Re: [PATCH v2 0/6] RFC: gup+dma: tracking dma-pinned pages

2018-11-20 Thread John Hubbard
's my main concern: > Hi Tom, Thanks again for looking at this! > On 11/10/2018 3:50 AM, john.hubb...@gmail.com wrote: >> From: John Hubbard >> ... >> -- >> WITHOUT the patch: >> --

Re: [PATCH v2 0/6] RFC: gup+dma: tracking dma-pinned pages

2018-11-20 Thread John Hubbard
's my main concern: > Hi Tom, Thanks again for looking at this! > On 11/10/2018 3:50 AM, john.hubb...@gmail.com wrote: >> From: John Hubbard >> ... >> -- >> WITHOUT the patch: >> --

Re: [PATCH v2 1/6] mm/gup: finish consolidating error handling

2018-11-14 Thread John Hubbard
On 11/12/18 8:14 AM, Dan Williams wrote: On Mon, Nov 12, 2018 at 7:45 AM Keith Busch wrote: On Sat, Nov 10, 2018 at 12:50:36AM -0800, john.hubb...@gmail.com wrote: From: John Hubbard An upcoming patch wants to be able to operate on each page that get_user_pages has retrieved. In order

Re: [PATCH v2 1/6] mm/gup: finish consolidating error handling

2018-11-14 Thread John Hubbard
On 11/12/18 8:14 AM, Dan Williams wrote: On Mon, Nov 12, 2018 at 7:45 AM Keith Busch wrote: On Sat, Nov 10, 2018 at 12:50:36AM -0800, john.hubb...@gmail.com wrote: From: John Hubbard An upcoming patch wants to be able to operate on each page that get_user_pages has retrieved. In order

[PATCH v2 6/6] mm: track gup pages with page->dma_pinned_* fields

2018-11-10 Thread john . hubbard
From: John Hubbard This patch sets and restores the new page->dma_pinned_flags and page->dma_pinned_count fields, but does not actually use them for anything yet. In order to use these fields at all, the page must be removed from any LRU list that it's on. The patch also adds some preca

[PATCH v2 6/6] mm: track gup pages with page->dma_pinned_* fields

2018-11-10 Thread john . hubbard
From: John Hubbard This patch sets and restores the new page->dma_pinned_flags and page->dma_pinned_count fields, but does not actually use them for anything yet. In order to use these fields at all, the page must be removed from any LRU list that it's on. The patch also adds some preca

[PATCH v2 5/6] mm: introduce zone_gup_lock, for dma-pinned pages

2018-11-10 Thread john . hubbard
From: John Hubbard The page->dma_pinned_flags and _count fields require lock protection. A lock at approximately the granularity of the zone_lru_lock is called for, but adding to the locking contention of zone_lru_lock is undesirable, because that is a pre-existing hot spot. Fortunat

[PATCH v2 5/6] mm: introduce zone_gup_lock, for dma-pinned pages

2018-11-10 Thread john . hubbard
From: John Hubbard The page->dma_pinned_flags and _count fields require lock protection. A lock at approximately the granularity of the zone_lru_lock is called for, but adding to the locking contention of zone_lru_lock is undesirable, because that is a pre-existing hot spot. Fortunat

[PATCH v2 4/6] mm: introduce page->dma_pinned_flags, _count

2018-11-10 Thread john . hubbard
From: John Hubbard Add two struct page fields that, combined, are unioned with struct page->lru. There is no change in the size of struct page. These new fields are for type safety and clarity. Also add page flag accessors to test, set and clear the new page->dma_pinned_flags field. Th

[PATCH v2 4/6] mm: introduce page->dma_pinned_flags, _count

2018-11-10 Thread john . hubbard
From: John Hubbard Add two struct page fields that, combined, are unioned with struct page->lru. There is no change in the size of struct page. These new fields are for type safety and clarity. Also add page flag accessors to test, set and clear the new page->dma_pinned_flags field. Th

[PATCH v2 0/6] RFC: gup+dma: tracking dma-pinned pages

2018-11-10 Thread john . hubbard
From: John Hubbard Hi, here is fodder for conversation during LPC. Changes since v1: a) Uses a simpler set/clear/test bit approach in the page flags. b) Added some initial performance results in the cover letter here, below. c) Rebased to latest linux.git d) Puts pages back on the LRU when

[PATCH v2 2/6] mm: introduce put_user_page*(), placeholder versions

2018-11-10 Thread john . hubbard
From: John Hubbard Introduces put_user_page(), which simply calls put_page(). This provides a way to update all get_user_pages*() callers, so that they call put_user_page(), instead of put_page(). Also introduces put_user_pages(), and a few dirty/locked variations, as a replacement

[PATCH v2 1/6] mm/gup: finish consolidating error handling

2018-11-10 Thread john . hubbard
From: John Hubbard An upcoming patch wants to be able to operate on each page that get_user_pages has retrieved. In order to do that, it's best to have a common exit point from the routine. Most of this has been taken care of by commit df06b37ffe5a4 ("mm/gup: cache dev_pagemap while pi

[PATCH v2 3/6] infiniband/mm: convert put_page() to put_user_page*()

2018-11-10 Thread john . hubbard
From: John Hubbard For infiniband code that retains pages via get_user_pages*(), release those pages via the new put_user_page(), or put_user_pages*(), instead of put_page() This is a tiny part of the second step of fixing the problem described in [1]. The steps are: 1) Provide put_user_page

[PATCH v2 0/6] RFC: gup+dma: tracking dma-pinned pages

2018-11-10 Thread john . hubbard
From: John Hubbard Hi, here is fodder for conversation during LPC. Changes since v1: a) Uses a simpler set/clear/test bit approach in the page flags. b) Added some initial performance results in the cover letter here, below. c) Rebased to latest linux.git d) Puts pages back on the LRU when

[PATCH v2 2/6] mm: introduce put_user_page*(), placeholder versions

2018-11-10 Thread john . hubbard
From: John Hubbard Introduces put_user_page(), which simply calls put_page(). This provides a way to update all get_user_pages*() callers, so that they call put_user_page(), instead of put_page(). Also introduces put_user_pages(), and a few dirty/locked variations, as a replacement

[PATCH v2 1/6] mm/gup: finish consolidating error handling

2018-11-10 Thread john . hubbard
From: John Hubbard An upcoming patch wants to be able to operate on each page that get_user_pages has retrieved. In order to do that, it's best to have a common exit point from the routine. Most of this has been taken care of by commit df06b37ffe5a4 ("mm/gup: cache dev_pagemap while pi

[PATCH v2 3/6] infiniband/mm: convert put_page() to put_user_page*()

2018-11-10 Thread john . hubbard
From: John Hubbard For infiniband code that retains pages via get_user_pages*(), release those pages via the new put_user_page(), or put_user_pages*(), instead of put_page() This is a tiny part of the second step of fixing the problem described in [1]. The steps are: 1) Provide put_user_page

Re: [PATCH 4/6] mm: introduce page->dma_pinned_flags, _count

2018-11-06 Thread John Hubbard
On 11/6/18 12:41 PM, Dave Chinner wrote: > On Tue, Nov 06, 2018 at 12:00:06PM +0100, Jan Kara wrote: >> On Tue 06-11-18 13:47:15, Dave Chinner wrote: >>> On Mon, Nov 05, 2018 at 04:26:04PM -0800, John Hubbard wrote: >>>> On 11/5/18 1:54 AM, Jan Kara wrote: >>&

Re: [PATCH 4/6] mm: introduce page->dma_pinned_flags, _count

2018-11-06 Thread John Hubbard
On 11/6/18 12:41 PM, Dave Chinner wrote: > On Tue, Nov 06, 2018 at 12:00:06PM +0100, Jan Kara wrote: >> On Tue 06-11-18 13:47:15, Dave Chinner wrote: >>> On Mon, Nov 05, 2018 at 04:26:04PM -0800, John Hubbard wrote: >>>> On 11/5/18 1:54 AM, Jan Kara wrote: >>&

Re: [PATCH 4/6] mm: introduce page->dma_pinned_flags, _count

2018-11-05 Thread John Hubbard
On 11/5/18 1:54 AM, Jan Kara wrote: > On Sun 04-11-18 23:10:12, John Hubbard wrote: >> On 10/13/18 9:47 AM, Christoph Hellwig wrote: >>> On Sat, Oct 13, 2018 at 12:34:12AM -0700, John Hubbard wrote: >>>> In patch 6/6, pin_page_for_dma(), which is called at t

Re: [PATCH 4/6] mm: introduce page->dma_pinned_flags, _count

2018-11-05 Thread John Hubbard
On 11/5/18 1:54 AM, Jan Kara wrote: > On Sun 04-11-18 23:10:12, John Hubbard wrote: >> On 10/13/18 9:47 AM, Christoph Hellwig wrote: >>> On Sat, Oct 13, 2018 at 12:34:12AM -0700, John Hubbard wrote: >>>> In patch 6/6, pin_page_for_dma(), which is called at t

Re: [PATCH v4 2/3] mm: introduce put_user_page*(), placeholder versions

2018-11-04 Thread John Hubbard
On 10/18/18 3:19 AM, Jan Kara wrote: > On Thu 11-10-18 20:53:34, John Hubbard wrote: >> On 10/11/18 6:23 PM, John Hubbard wrote: >>> On 10/11/18 6:20 AM, Jason Gunthorpe wrote: >>>> On Thu, Oct 11, 2018 at 10:49:29AM +0200, Jan Kara wrote: [...] > Well, put_pag

Re: [PATCH v4 2/3] mm: introduce put_user_page*(), placeholder versions

2018-11-04 Thread John Hubbard
On 10/18/18 3:19 AM, Jan Kara wrote: > On Thu 11-10-18 20:53:34, John Hubbard wrote: >> On 10/11/18 6:23 PM, John Hubbard wrote: >>> On 10/11/18 6:20 AM, Jason Gunthorpe wrote: >>>> On Thu, Oct 11, 2018 at 10:49:29AM +0200, Jan Kara wrote: [...] > Well, put_pag

Re: [PATCH v4 2/3] mm: introduce put_user_page*(), placeholder versions

2018-11-04 Thread John Hubbard
On 10/22/18 12:43 PM, Jason Gunthorpe wrote: > On Thu, Oct 11, 2018 at 06:23:24PM -0700, John Hubbard wrote: >> On 10/11/18 6:20 AM, Jason Gunthorpe wrote: >>> On Thu, Oct 11, 2018 at 10:49:29AM +0200, Jan Kara wrote: >>> >>>>> This is a real worry. If s

Re: [PATCH v4 2/3] mm: introduce put_user_page*(), placeholder versions

2018-11-04 Thread John Hubbard
On 10/22/18 12:43 PM, Jason Gunthorpe wrote: > On Thu, Oct 11, 2018 at 06:23:24PM -0700, John Hubbard wrote: >> On 10/11/18 6:20 AM, Jason Gunthorpe wrote: >>> On Thu, Oct 11, 2018 at 10:49:29AM +0200, Jan Kara wrote: >>> >>>>> This is a real worry. If s

Re: [PATCH 4/6] mm: introduce page->dma_pinned_flags, _count

2018-11-04 Thread John Hubbard
On 10/13/18 9:47 AM, Christoph Hellwig wrote: > On Sat, Oct 13, 2018 at 12:34:12AM -0700, John Hubbard wrote: >> In patch 6/6, pin_page_for_dma(), which is called at the end of >> get_user_pages(), >> unceremoniously rips the pages out of the LRU, as a prerequisite to using

Re: [PATCH 4/6] mm: introduce page->dma_pinned_flags, _count

2018-11-04 Thread John Hubbard
On 10/13/18 9:47 AM, Christoph Hellwig wrote: > On Sat, Oct 13, 2018 at 12:34:12AM -0700, John Hubbard wrote: >> In patch 6/6, pin_page_for_dma(), which is called at the end of >> get_user_pages(), >> unceremoniously rips the pages out of the LRU, as a prerequisite to using

Re: [PATCH 4/6] mm: introduce page->dma_pinned_flags, _count

2018-11-02 Thread John Hubbard
On 10/24/18 4:00 AM, Balbir Singh wrote: > On Fri, Oct 12, 2018 at 05:15:51PM -0700, John Hubbard wrote: >> On 10/12/18 3:56 AM, Balbir Singh wrote: >>> On Thu, Oct 11, 2018 at 11:00:12PM -0700, john.hubb...@gmail.com wrote: >>>> From: John Hubbard >> [...] >

Re: [PATCH 4/6] mm: introduce page->dma_pinned_flags, _count

2018-11-02 Thread John Hubbard
On 10/24/18 4:00 AM, Balbir Singh wrote: > On Fri, Oct 12, 2018 at 05:15:51PM -0700, John Hubbard wrote: >> On 10/12/18 3:56 AM, Balbir Singh wrote: >>> On Thu, Oct 11, 2018 at 11:00:12PM -0700, john.hubb...@gmail.com wrote: >>>> From: John Hubbard >> [...] >

Re: [PATCH 4/6] mm: introduce page->dma_pinned_flags, _count

2018-10-17 Thread John Hubbard
On 10/17/18 4:09 AM, Michal Hocko wrote: > On Tue 16-10-18 18:48:23, John Hubbard wrote: > [...] >> It's hard to say exactly what the active/inactive/unevictable list should >> be when DMA is done and put_user_page*() is called, because we don't know >> if some device rea

Re: [PATCH 4/6] mm: introduce page->dma_pinned_flags, _count

2018-10-17 Thread John Hubbard
On 10/17/18 4:09 AM, Michal Hocko wrote: > On Tue 16-10-18 18:48:23, John Hubbard wrote: > [...] >> It's hard to say exactly what the active/inactive/unevictable list should >> be when DMA is done and put_user_page*() is called, because we don't know >> if some device rea

Re: [PATCH 4/6] mm: introduce page->dma_pinned_flags, _count

2018-10-16 Thread John Hubbard
On 10/16/18 1:51 AM, Jan Kara wrote: > On Sun 14-10-18 10:01:24, Dave Chinner wrote: >> On Sat, Oct 13, 2018 at 12:34:12AM -0700, John Hubbard wrote: >>> On 10/12/18 8:55 PM, Dave Chinner wrote: >>>> On Thu, Oct 11, 2018 at 11:00:12PM -0700, john.hubb...@gmail.com w

Re: [PATCH 4/6] mm: introduce page->dma_pinned_flags, _count

2018-10-16 Thread John Hubbard
On 10/16/18 1:51 AM, Jan Kara wrote: > On Sun 14-10-18 10:01:24, Dave Chinner wrote: >> On Sat, Oct 13, 2018 at 12:34:12AM -0700, John Hubbard wrote: >>> On 10/12/18 8:55 PM, Dave Chinner wrote: >>>> On Thu, Oct 11, 2018 at 11:00:12PM -0700, john.hubb...@gmail.com w

Re: [PATCH 4/6] mm: introduce page->dma_pinned_flags, _count

2018-10-13 Thread John Hubbard
On 10/13/18 9:47 AM, Christoph Hellwig wrote: > On Sat, Oct 13, 2018 at 12:34:12AM -0700, John Hubbard wrote: >> In patch 6/6, pin_page_for_dma(), which is called at the end of >> get_user_pages(), >> unceremoniously rips the pages out of the LRU, as a prerequisite to using

Re: [PATCH 4/6] mm: introduce page->dma_pinned_flags, _count

2018-10-13 Thread John Hubbard
On 10/13/18 9:47 AM, Christoph Hellwig wrote: > On Sat, Oct 13, 2018 at 12:34:12AM -0700, John Hubbard wrote: >> In patch 6/6, pin_page_for_dma(), which is called at the end of >> get_user_pages(), >> unceremoniously rips the pages out of the LRU, as a prerequisite to using

Re: [PATCH 4/6] mm: introduce page->dma_pinned_flags, _count

2018-10-13 Thread John Hubbard
On 10/12/18 8:55 PM, Dave Chinner wrote: > On Thu, Oct 11, 2018 at 11:00:12PM -0700, john.hubb...@gmail.com wrote: >> From: John Hubbard [...] >> diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h >> index 5ed8f6292a53..017ab82e36ca 100644 >> --- a/includ

Re: [PATCH 4/6] mm: introduce page->dma_pinned_flags, _count

2018-10-13 Thread John Hubbard
On 10/12/18 8:55 PM, Dave Chinner wrote: > On Thu, Oct 11, 2018 at 11:00:12PM -0700, john.hubb...@gmail.com wrote: >> From: John Hubbard [...] >> diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h >> index 5ed8f6292a53..017ab82e36ca 100644 >> --- a/includ

Re: [PATCH 6/6] mm: track gup pages with page->dma_pinned_* fields

2018-10-12 Thread John Hubbard
On 10/12/18 4:07 AM, Balbir Singh wrote: > On Thu, Oct 11, 2018 at 11:00:14PM -0700, john.hubb...@gmail.com wrote: >> From: John Hubbard [...] >> +static int pin_page_for_dma(struct page *page) >> +{ >> +int ret = 0; >> +struct zone *zone; >&

Re: [PATCH 6/6] mm: track gup pages with page->dma_pinned_* fields

2018-10-12 Thread John Hubbard
On 10/12/18 4:07 AM, Balbir Singh wrote: > On Thu, Oct 11, 2018 at 11:00:14PM -0700, john.hubb...@gmail.com wrote: >> From: John Hubbard [...] >> +static int pin_page_for_dma(struct page *page) >> +{ >> +int ret = 0; >> +struct zone *zone; >&

<    3   4   5   6   7   8   9   10   11   12   >