Re: [PATCH 1/1] ipc/shm: fix the historical/wrong mm->start_stack check

2014-08-25 Thread Hugh Dickins
On Mon, 25 Aug 2014, Oleg Nesterov wrote: > On 08/24, Hugh Dickins wrote: > > > > I'd say it comes earlier, from Christoph Rohland's 2.4.17-pre7's > > "Add missing checks on shmat()", though I didn't find more than that. > > > > We can all understand

Re: [PATCH 1/1] ipc/shm: fix the historical/wrong mm->start_stack check

2014-08-25 Thread Hugh Dickins
On Mon, 25 Aug 2014, Oleg Nesterov wrote: > On 08/25, Hugh Dickins wrote: > > > And I think I'll let Linus's guard page justify your 4 (to match comment) > > in place of the original's mysterious 5. > > Ah, thanks again. Yes, if we want to guarantee 4 pages we shoul

Re: [PATCH v2] ipc/shm: fix the historical/wrong mm->start_stack check

2014-08-25 Thread Hugh Dickins
On Mon, 25 Aug 2014, Oleg Nesterov wrote: > The ->start_stack check in do_shmat() looks ugly and simply wrong. > > 1. ->start_stack is only valid right after exec(), the application >can switch to another stack and even unmap this area. Or a stack >can simply grow, ->start_stack won't

Re: 3.17-rc1: leds blink workqueue causes sleeping BUGs

2014-08-25 Thread Hugh Dickins
On Mon, 25 Aug 2014, Samuel Thibault wrote: > Samuel Thibault, le Mon 25 Aug 2014 23:23:24 +0200, a écrit : > > We could indeed have a loop if the user was making the VT::* leds use > > the vt-* trigger, > > Actually, while there can be a loop, it wouldn't be possible to inject > events in it: a

Re: [PATCH v5] mm: softdirty: enable write notifications on VMAs after VM_SOFTDIRTY cleared

2014-08-25 Thread Hugh Dickins
On Sun, 24 Aug 2014, Peter Feiner wrote: > For VMAs that don't want write notifications, PTEs created for read > faults have their write bit set. If the read fault happens after > VM_SOFTDIRTY is cleared, then the PTE's softdirty bit will remain > clear after subsequent writes. Good catch.

Re: [PATCH v3] ipc/shm: kill the historical/wrong mm->start_stack check

2014-08-26 Thread Hugh Dickins
This means > that in the likely case "addr > start_stack - size - PAGE_SIZE * 5" > is simply impossible after find_vma_intersection() == F, or the stack > can't grow anyway because of RLIMIT_STACK. > > Many thanks to Hugh for his explanations. > > Signed-off-by: Oleg Nes

Re: [PATCH v5] mm: softdirty: enable write notifications on VMAs after VM_SOFTDIRTY cleared

2014-08-27 Thread Hugh Dickins
On Tue, 26 Aug 2014, Cyrill Gorcunov wrote: > On Mon, Aug 25, 2014 at 09:45:34PM -0700, Hugh Dickins wrote: > > > > Hmm. For a long time I thought you were fixing another important bug > > with down_write, since we "always" use down_write to modify vm_flags.

Re: [PATCH v5] mm: softdirty: enable write notifications on VMAs after VM_SOFTDIRTY cleared

2014-08-27 Thread Hugh Dickins
On Tue, 26 Aug 2014, Cyrill Gorcunov wrote: > On Tue, Aug 26, 2014 at 06:43:55PM +0300, Kirill A. Shutemov wrote: > > On Tue, Aug 26, 2014 at 07:18:13PM +0400, Cyrill Gorcunov wrote: > > > > Basically, it's safe if only soft-dirty is allowed to modify vm_flags > > > > without down_write(). But why

Re: [PATCH v4 2/2] ksm: provide support to use deferrable timers for scanner thread

2014-08-28 Thread Hugh Dickins
I'm rather hoping we can strike a good enough balance with your deferrable timer, that nobody will need any better. So, with a few changes here and below, please add my Acked-by: Hugh Dickins to patches 1 and 2, and resend to akpm - thank you! Here (above), it's restore the text to V3's To

Re: mm: BUG in unmap_page_range

2014-09-10 Thread Hugh Dickins
On Wed, 10 Sep 2014, Sasha Levin wrote: > On 09/09/2014 10:45 PM, Hugh Dickins wrote: > > Sasha, you say you're getting plenty of these now, but I've only seen > > the dump for one of them, on Aug26: please post a few more dumps, so > > that we can look for commonality. >

Re: mm: BUG in unmap_page_range

2014-09-10 Thread Hugh Dickins
On Wed, 10 Sep 2014, Mel Gorman wrote: > On Tue, Sep 09, 2014 at 07:45:26PM -0700, Hugh Dickins wrote: > > > > I've been rather assuming that the 9d340902 seen in many of the > > registers in that Aug26 dump is the pte val in question: that's > > SOFT_DIRTY|PROTNONE|

Re: mm: BUG in unmap_page_range

2014-09-10 Thread Hugh Dickins
On Wed, 10 Sep 2014, Sasha Levin wrote: > On 09/10/2014 03:09 PM, Hugh Dickins wrote: > > Thanks for supplying, but the change in inlining means that > > change_protection_range() and change_protection() are no longer > > relevant for these traces, we now need to see change_pt

Re: mm: BUG in unmap_page_range

2014-09-11 Thread Hugh Dickins
On Wed, 10 Sep 2014, Sasha Levin wrote: > On 09/10/2014 03:36 PM, Hugh Dickins wrote: > > Right, and Sasha reports that that can fire, but he sees the bug > > with this patch in and without that firing. > > I've changed that WARN_ON_ONCE() to a VM_BUG_ON_VMA() to

Re: [PATCH v4 2/2] ksm: provide support to use deferrable timers for scanner thread

2014-09-11 Thread Hugh Dickins
xt_switch() anyway. [PATCH v2] ksm: avoid periodic wakeup while mergeable mms are quiet Description yet to be written! Reported-by: Chintan Pandya Not-Signed-off-by: Hugh Dickins --- include/linux/ksm.h | 11 + include/linux/sched.h |1 kernel/sched/core.c |

Re: [PATCH v4 2/2] ksm: provide support to use deferrable timers for scanner thread

2014-09-11 Thread Hugh Dickins
On Wed, 10 Sep 2014, Peter Zijlstra wrote: > > Does it make sense to drive both KSM and khugepage the same way we drive > the numa scanning? It has the benefit of getting rid of these threads, > which pushes the work into the right accountable context (the task its > doing the scanning for) and

Re: [PATCH v4 2/2] ksm: provide support to use deferrable timers for scanner thread

2014-09-11 Thread Hugh Dickins
On Thu, 11 Sep 2014, Chintan Pandya wrote: > I don't mean to divert the thread too much. But just one suggestion offered > by Harshad. > > Why can't we stop invoking more of a KSM scanner thread when we are > saturating from savings ? But again, to check whether savings are saturated > or not,

Re: [PATCH v3 2/6] mm/hugetlb: take page table lock in follow_huge_(addr|pmd|pud)()

2014-09-08 Thread Hugh Dickins
On Fri, 5 Sep 2014, Naoya Horiguchi wrote: > On Wed, Sep 03, 2014 at 02:17:41PM -0700, Hugh Dickins wrote: > > On Thu, 28 Aug 2014, Naoya Horiguchi wrote: > > > > > > Reported-by: Hugh Dickins > > > Signed-off-by: Naoya Horiguchi > > > Cc: # [3.12+

Re: [PATCH v4 2/2] ksm: provide support to use deferrable timers for scanner thread

2014-09-08 Thread Hugh Dickins
On Wed, 3 Sep 2014, Peter Zijlstra wrote: > On Wed, Aug 27, 2014 at 11:02:20PM -0700, Hugh Dickins wrote: > > On Wed, 20 Aug 2014, Chintan Pandya wrote: > > > > > KSM thread to scan pages is scheduled on definite timeout. That wakes up > > > CPU from idle sta

Re: [PATCH v3 2/6] mm/hugetlb: take page table lock in follow_huge_(addr|pmd|pud)()

2014-09-09 Thread Hugh Dickins
On Mon, 8 Sep 2014, Naoya Horiguchi wrote: > On Mon, Sep 08, 2014 at 12:13:16AM -0700, Hugh Dickins wrote: > > On Fri, 5 Sep 2014, Naoya Horiguchi wrote: > > > On Wed, Sep 03, 2014 at 02:17:41PM -0700, Hugh Dickins wrote: > > > > > > > One subtlety to take

Re: [PATCH v4 2/2] ksm: provide support to use deferrable timers for scanner thread

2014-09-09 Thread Hugh Dickins
On Mon, 8 Sep 2014, Peter Zijlstra wrote: > On Mon, Sep 08, 2014 at 01:25:36AM -0700, Hugh Dickins wrote: > > > > --- 3.17-rc4/include/linux/ksm.h2014-03-30 20:40:15.0 -0700 > > +++ linux/include/linux/ksm.h 2014-09-07 11:54:41.528003316 -0700 > >

Re: [PATCH v4 2/2] ksm: provide support to use deferrable timers for scanner thread

2014-09-09 Thread Hugh Dickins
pers grow increasingly sick and sceptical of such knobs, preferring to make an effort to get things working well without them. Both attitudes are valid. > > > On Mon, Sep 08, 2014 at 01:25:36AM -0700, Hugh Dickins wrote: > > > Well, yes, but... how do we know when there is no mor

Re: mm: BUG in unmap_page_range

2014-09-09 Thread Hugh Dickins
On Tue, 9 Sep 2014, Sasha Levin wrote: > On 09/09/2014 05:33 PM, Mel Gorman wrote: > > On Mon, Sep 08, 2014 at 01:56:55PM -0400, Sasha Levin wrote: > >> On 09/08/2014 01:18 PM, Mel Gorman wrote: > >>> A worse possibility is that somehow the lock is getting corrupted but > >>> that's also a tough

Re: [PATCH v2 1/3] mm/hugetlb: take refcount under page table lock in follow_huge_pmd()

2014-08-09 Thread Hugh Dickins
On Fri, 1 Aug 2014, Naoya Horiguchi wrote: > We have a race condition between move_pages() and freeing hugepages, > where move_pages() calls follow_page(FOLL_GET) for hugepages internally > and tries to get its refcount without preventing concurrent freeing. > This race crashes the kernel, so

Re: [PATCH v2 2/3] mm/hugetlb: use get_page_unless_zero() in hugetlb_fault()

2014-08-09 Thread Hugh Dickins
On Fri, 1 Aug 2014, Naoya Horiguchi wrote: > After fixing locking in follow_page(FOLL_GET) for hugepages, I start to > observe the BUG of "get_page() on refcount 0 page" in hugetlb_fault() in > the same test. > > I'm not exactly sure about how this race is triggered, but hugetlb_fault() > calls

Re: [PATCH v2 3/3] mm/hugetlb: add migration entry check in hugetlb_change_protection

2014-08-09 Thread Hugh Dickins
On Fri, 1 Aug 2014, Naoya Horiguchi wrote: > There is a race condition between hugepage migration and change_protection(), > where hugetlb_change_protection() doesn't care about migration entries and > wrongly overwrites them. That causes unexpected results like kernel crash. > > This patch adds

Re: [PATCH v3 2/2] ksm: provide support to use deferrable timers for scanner thread

2014-08-11 Thread Hugh Dickins
rence). I expect it to need the same treatment. Hugh > > Signed-off-by: Chintan Pandya > Cc: Thomas Gleixner > Cc: John Stultz > Cc: Peter Zijlstra > Cc: Ingo Molnar > Cc: Hugh Dickins > --- > Changes: > > V2-->V3: > - Handled error case properly

Re: [PATCH 05/16] page-flags: define behavior of FS/IO-related flags on compound pages

2015-03-22 Thread Hugh Dickins
On Thu, 19 Mar 2015, Kirill A. Shutemov wrote: > On Thu, Mar 19, 2015 at 11:29:52AM -0700, Dave Hansen wrote: > > On 03/19/2015 10:08 AM, Kirill A. Shutemov wrote: > > > The odd exception is PG_dirty: sound uses compound pages and maps them > > > with PTEs. NO_COMPOUND triggers VM_BUG_ON() in

Re: [PATCH 01/16] mm: consolidate all page-flags helpers in

2015-03-22 Thread Hugh Dickins
On Thu, 19 Mar 2015, Kirill A. Shutemov wrote: > We have page-flags helper function declarations/definitions spread over > several header files. Let's consolidate them in . > > Signed-off-by: Kirill A. Shutemov Acked-by: Hugh Dickins I find this one helpful (assuming it

Re: [PATCH 02/16] page-flags: trivial cleanup for PageTrans* helpers

2015-03-22 Thread Hugh Dickins
On Thu, 19 Mar 2015, Kirill A. Shutemov wrote: > Use TESTPAGEFLAG_FALSE() to get it a bit cleaner. > > Signed-off-by: Kirill A. Shutemov Yeah, this is okay too. > --- > include/linux/page-flags.h | 18 +++--- > 1 file changed, 3 insertions(+), 15 deletions(-) > > diff --git

Re: [PATCH 00/16] Sanitize usage of ->flags and ->mapping for tail pages

2015-03-22 Thread Hugh Dickins
On Thu, 19 Mar 2015, Kirill A. Shutemov wrote: > Currently we take naive approach to page flags on compound -- we set the > flag on the page without consideration if the flag makes sense for tail > page or for compound page in general. This patchset try to sort this out > by defining per-flag

Re: [PATCH 00/24] huge tmpfs: an alternative approach to THPageCache

2015-03-22 Thread Hugh Dickins
On Mon, 23 Feb 2015, Kirill A. Shutemov wrote: > > I scanned through the patches to get general idea on how it works. Thanks! > I'm not > sure that I will have time and will power to do proper code-digging before > the summit. I found few bugs in my patchset which I want to troubleshoot >

Re: [PATCH 01/24] mm: update_lru_size warn and reset bad lru_size

2015-03-22 Thread Hugh Dickins
On Mon, 23 Feb 2015, Kirill A. Shutemov wrote: > On Fri, Feb 20, 2015 at 07:51:16PM -0800, Hugh Dickins wrote: > > Though debug kernels have a VM_BUG_ON to help protect from misaccounting > > lru_size, non-debug kernels are liable to wrap it around: and then the > > vast uns

Re: [PATCH 03/24] mm: use __SetPageSwapBacked and don't ClearPageSwapBacked

2015-03-22 Thread Hugh Dickins
On Wed, 25 Feb 2015, Mel Gorman wrote: > On Fri, Feb 20, 2015 at 07:56:15PM -0800, Hugh Dickins wrote: > > Commit 07a427884348 ("mm: shmem: avoid atomic operation during > > shmem_getpage_gfp") rightly replaced one instance of SetPageSwapBacked > > by __

Re: [PATCH 11/24] huge tmpfs: shrinker to migrate and free underused holes

2015-03-22 Thread Hugh Dickins
On Thu, 19 Mar 2015, Konstantin Khlebnikov wrote: > On 21.02.2015 07:09, Hugh Dickins wrote: > > > > The "team_usage" field added to struct page (in union with "private") > > is somewhat vaguely named: because while the huge page is sparsely > &

Re: [PATCH 05/16] page-flags: define behavior of FS/IO-related flags on compound pages

2015-03-24 Thread Hugh Dickins
On Mon, 23 Mar 2015, Kirill A. Shutemov wrote: > On Sun, Mar 22, 2015 at 05:02:58PM -0700, Hugh Dickins wrote: > > On Thu, 19 Mar 2015, Kirill A. Shutemov wrote: > > > On Thu, Mar 19, 2015 at 11:29:52AM -0700, Dave Hansen wrote: > > > > On 03/19/2015 10:0

Re: [PATCH 00/16] Sanitize usage of ->flags and ->mapping for tail pages

2015-03-24 Thread Hugh Dickins
On Mon, 23 Mar 2015, Kirill A. Shutemov wrote: > On Sun, Mar 22, 2015 at 05:28:47PM -0700, Hugh Dickins wrote: > > On Thu, 19 Mar 2015, Kirill A. Shutemov wrote: > > > > > Currently we take naive approach to page flags on compound -- we set the > > > flag

Re: [PATCH 11/24] huge tmpfs: shrinker to migrate and free underused holes

2015-03-24 Thread Hugh Dickins
On Tue, 24 Mar 2015, Kirill A. Shutemov wrote: > On Sun, Mar 22, 2015 at 09:40:02PM -0700, Hugh Dickins wrote: > > (I think Kirill has a problem of that kind in his page_remove_rmap scan). (And this one I mentioned to you at the conference :) > > > > It will be interesti

Re: [PATCH 05/16] page-flags: define behavior of FS/IO-related flags on compound pages

2015-03-25 Thread Hugh Dickins
On Wed, 25 Mar 2015, Kirill A. Shutemov wrote: > > We only need tail refcounting for THP, so I think this should fix the issue: > > diff --git a/include/linux/mm.h b/include/linux/mm.h > index 4a3a38522ab4..9ab432660adb 100644 > --- a/include/linux/mm.h > +++ b/include/linux/mm.h > @@ -456,7

Re: [PATCH] mm: avoid tail page refcounting on non-THP compound pages

2015-03-25 Thread Hugh Dickins
e about anything which can lead to > crash or other serious misbehaviour. > > Since currently all THP pages are anonymous and all drivers pages are > not, we can fix the __compound_tail_refcounted() check by requiring > PageAnon() to enable tail page refcounting. > > Signed-

Re: [PATCH] mm: avoid tail page refcounting on non-THP compound pages

2015-03-25 Thread Hugh Dickins
On Wed, 25 Mar 2015, Hugh Dickins wrote: > On Thu, 26 Mar 2015, Kirill A. Shutemov wrote: > > > THP uses tail page refcounting to be able to split huge page at any > > time. Tail page refcounting is not needed for rest users of compound > > pages and it's harmful because

Re: [PATCH] mm: avoid tail page refcounting on non-THP compound pages

2015-03-25 Thread Hugh Dickins
On Thu, 26 Mar 2015, Kirill A. Shutemov wrote: > On Wed, Mar 25, 2015 at 03:48:48PM -0700, Hugh Dickins wrote: > > On Wed, 25 Mar 2015, Hugh Dickins wrote: > > > On Thu, 26 Mar 2015, Kirill A. Shutemov wrote: > > > > > > > > Since currently all THP

Re: [patch][resend] MAP_HUGETLB munmap fails with size not 2MB aligned

2015-03-25 Thread Hugh Dickins
On Wed, 22 Oct 2014, Davide Libenzi wrote: > [Resending with proper CC list suggested by Andrew] I have recently been reminded of this languishing in my inbox ;) (along with many others that I won't get to answer so quickly). And in turn it reminds me of an older from Joern, who was annoyed

Re: [PATCHv4 12/24] thp: PMD splitting without splitting compound page

2015-04-01 Thread Hugh Dickins
On Wed, 1 Apr 2015, Kirill A. Shutemov wrote: > On Wed, Apr 01, 2015 at 12:08:35PM +0530, Aneesh Kumar K.V wrote: > > > > With this we now have pte mapping part of a compound page(). Now the > > gneric gup implementation does > > > > gup_pte_range() > > ptem = ptep = pte_offset_map(, addr);

Re: [patch -mm] mm, doc: cleanup and clarify munmap behavior for hugetlb memory fix

2015-04-02 Thread Hugh Dickins
David Rientjes Thanks, yes, good wording: it is best to be a bit vague here, since each msyscall takes the approach most convenient for it. Acked-by: Hugh Dickins > --- > Documentation/vm/hugetlbpage.txt | 7 +-- > 1 file changed, 5 insertions(+), 2 deletions(-) > >

Re: arm/ksm: Unable to handle kernel paging request in get_ksm_page() and ksm_scan_thread()

2015-03-29 Thread Hugh Dickins
On Sat, 28 Mar 2015, Xishi Qiu wrote: > On 2015/3/26 21:23, Xishi Qiu wrote: > > > Here are two panic logs from smart phone test, and the kernel version is > > v3.10. > > > > log1 is "Unable to handle kernel paging request at virtual address > > c0704da020", it should be ffc0704da020,

Re: [patch 1/2] mm, doc: cleanup and clarify munmap behavior for hugetlb memory

2015-03-29 Thread Hugh Dickins
On Thu, 26 Mar 2015, David Rientjes wrote: > munmap(2) of hugetlb memory requires a length that is hugepage aligned, > otherwise it may fail. Add this to the documentation. Thanks for taking this on, David. But although munmap(2) is the one Davide called out, it goes beyond that, doesn't it?

Re: arm/ksm: Unable to handle kernel paging request in get_ksm_page() and ksm_scan_thread()

2015-03-29 Thread Hugh Dickins
On Mon, 30 Mar 2015, Xishi Qiu wrote: > On 2015/3/30 9:46, Xishi Qiu wrote: > > On 2015/3/30 8:43, Hugh Dickins wrote: > >> On Sat, 28 Mar 2015, Xishi Qiu wrote: > >>> On 2015/3/26 21:23, Xishi Qiu wrote: > >>> > >>>> Here are two

Re: [patch 1/2] mm, doc: cleanup and clarify munmap behavior for hugetlb memory

2015-03-30 Thread Hugh Dickins
On Mon, 30 Mar 2015, Eric B Munson wrote: > On Sun, 29 Mar 2015, Hugh Dickins wrote: > > > > Eric, I apologize for bringing you in to the discussion, and then > > ignoring your input. I understand that you would like MAP_HUGETLB > > to behave more unders

Re: [patch][resend] MAP_HUGETLB munmap fails with size not 2MB aligned

2015-03-30 Thread Hugh Dickins
touched by the munmap() request should be unmapped. > > > > Please include > > Acked-by: Eric B Munson > > to the original patch. I would like to see the mmap man page adjusted > > to make note of this behavior as well. > > This is just a bug fix and I never

Re: [PATCH 4/4] mm: make every pte dirty on do_swap_page

2015-04-11 Thread Hugh Dickins
htforward. Inherently, swapped-out page was > pte_dirty so this patch restores the dirtiness when swap-in fault > happens so madvise_free doesn't rely on the PageDirty any more. > > Cc: Hugh Dickins > Cc: Cyrill Gorcunov > Cc: Pavel Emelyanov > Reported-by: Yalin Wang > Signe

Re: [PATCH v2] ipc: Use private shmem or hugetlbfs inodes for shm segments.

2015-07-27 Thread Hugh Dickins
: [] lock_acquire+0xc7/0x270 > Jul 22 14:36:40 fc23 kernel: [] ? shm_close+0x34/0x130 > Jul 22 14:36:40 fc23 kernel: [] down_write+0x5a/0xc0 > Jul 22 14:36:40 fc23 kernel: [] ? shm_close+0x34/0x130 > Jul 22 14:36:40 fc23 kernel: [] shm_close+0x34/0x130 > Jul 22 14:36:40 fc23 kernel: []

Re: [PATCH v2] ipc: Use private shmem or hugetlbfs inodes for shm segments.

2015-07-27 Thread Hugh Dickins
On Mon, 27 Jul 2015, Stephen Smalley wrote: > On 07/27/2015 03:32 PM, Hugh Dickins wrote: > > On Fri, 24 Jul 2015, Stephen Smalley wrote: > >> --- a/fs/hugetlbfs/inode.c > >> +++ b/fs/hugetlbfs/inode.c > >> @@ -1010,6 +1010,8 @@ struct file *hugetlb_file_setup

Re: [PATCH] mm, vmscan: Do not wait for page writeback for GFP_NOFS allocations

2015-08-04 Thread Hugh Dickins
On Tue, 4 Aug 2015, Michal Hocko wrote: > On Mon 03-08-15 23:32:00, Hugh Dickins wrote: > [...] > > But I have modified it a little, I don't think you'll mind. As you > > suggested yourself, I actually prefer to test may_enter_fs there, rather > > than __GFP_FS: not

[PATCH] mm, vmscan: Do not wait for page writeback for GFP_NOFS allocations

2015-08-04 Thread Hugh Dickins
ause it is not : safe to do so, memcg reclaim or otherwise. Cc: sta...@vger.kernel.org # 3.9+ [ty...@mit.edu: corrected the control flow] Fixes: c3b94f44fcb0 ("memcg: further prevent OOM with too many dirty pages") Reported-by: Nikolay Borisov Signed-off-by: Michal Hocko Signed-off-by

page-flags behavior on compound pages: a worry

2015-08-05 Thread Hugh Dickins
Hi Kirill, I had a nasty thought this morning. Andrew had prodded me gently to re-examine my concerns with your page-flags rework in mmotm. I still dislike the bloat (my mm/built-in.o text goes up from 478513 to 490183 bytes on a non-DEBUG_VM build); but I was hoping to set that aside, to let

Re: page-flags behavior on compound pages: a worry

2015-08-06 Thread Hugh Dickins
On Thu, 6 Aug 2015, Kirill A. Shutemov wrote: > On Wed, Aug 05, 2015 at 09:15:57PM -0700, Hugh Dickins wrote: > > Hi Kirill, > > > > I had a nasty thought this morning. > > Tough day. > > I'm trying to wrap my head around this mail and not sure if I succeed

v4.2-rc dcache regression, probably 75a6f82a0d10

2015-07-31 Thread Hugh Dickins
I think there's something not quite right with the fs/dcache.c commit 75a6f82a0d10 ("freeing unlinked file indefinitely delayed"). When running my old tmpfs swapping load (two repetitive make -j20 kernel builds, one on tmpfs, one on ext4 over loop over tmpfs file, in limited memory with plenty of

Re: v4.2-rc dcache regression, probably 75a6f82a0d10

2015-07-31 Thread Hugh Dickins
On Fri, 31 Jul 2015, Linus Torvalds wrote: > On Fri, Jul 31, 2015 at 10:46 AM, Hugh Dickins wrote: > > > > Sounds like a dcache problem, and 75a6f82a0d10 seemed the only > > likely candidate, so I experimented with reverting it yesterday, > > and ran successfully for

Re: mm: shmem_zero_setup skip security check and lockdep conflict with XFS

2015-07-09 Thread Hugh Dickins
On Wed, 8 Jul 2015, Stephen Smalley wrote: > On 07/08/2015 09:13 AM, Stephen Smalley wrote: > > On Sun, Jun 14, 2015 at 12:48 PM, Hugh Dickins wrote: > >> It appears that, at some point last year, XFS made directory handling > >> changes which bring it into lockdep con

Re: v4.2-rc dcache regression, probably 75a6f82a0d10

2015-07-31 Thread Hugh Dickins
On Fri, 31 Jul 2015, Dominique Martinet wrote: > Hugh Dickins wrote on Fri, Jul 31, 2015: > > It will indeed be weird and odd if it confirms that DCACHE_DISCONNECTED > > revert is good. I agree that Dominique's 4bf46a272647 seems now more > > likely, if still unlikely;

Re: v4.2-rc dcache regression, probably 75a6f82a0d10

2015-07-31 Thread Hugh Dickins
On Fri, 31 Jul 2015, Linus Torvalds wrote: > > I'd be more suspicious about other effects. For example, iot's not at > all obvious that the commit in question just changes the order of the > flags/inode field accesses, there are potentialy bigger changes there. > For example, this part (in

Re: v4.2-rc dcache regression, probably 75a6f82a0d10

2015-07-31 Thread Hugh Dickins
On Fri, 31 Jul 2015, Hugh Dickins wrote: > On Fri, 31 Jul 2015, Linus Torvalds wrote: > > > > So leave it running a while longer, but maybe it's 4bf46a272647 like > > Dominique suspects. Although I don't see how that could trigger > > anything either.. > > I re

Re: [git pull] vfs.git spurious ENOTDIR fix

2015-08-01 Thread Hugh Dickins
On Sat, 1 Aug 2015, Linus Torvalds wrote: > > Well, I'd not be against continuing cleanups for 4.3... Well, as long > as we can make sure 4.2 is solid first, of course. I'd still like to > have Hugh verify that the current -git tree works for his load, but > obviously that wasn't easily

Re: [git pull] vfs.git spurious ENOTDIR fix

2015-08-02 Thread Hugh Dickins
On Sat, 1 Aug 2015, Linus Torvalds wrote: > On Sat, Aug 1, 2015 at 9:06 PM, Hugh Dickins wrote: > > > > (I don't actually understand why the clearing of DCACHE_ENTRY_TYPE in > > dentry_iput() is not of continuing concern; but don't worry, there's > > plenty I don't und

Re: [PATCH] mm, vmscan: Do not wait for page writeback for GFP_NOFS allocations

2015-08-04 Thread Hugh Dickins
context because it is not : safe to do so, memcg reclaim or otherwise. Cc: sta...@vger.kernel.org # 3.9+ [ty...@mit.edu: corrected the control flow] Fixes: c3b94f44fcb0 ("memcg: further prevent OOM with too many dirty pages") Reported-by: Nikolay Borisov Signed-off-by: Michal Hocko Signed-off-by: Hu

Re: [PATCH 3/3] btrfs: set FS_SUPPORTS_SEEK_HOLE flag.

2015-04-21 Thread Hugh Dickins
On Tue, 21 Apr 2015, Christoph Hellwig wrote: > On Mon, Apr 20, 2015 at 01:58:35PM +0100, Al Viro wrote: > > > > Do ext4 and xfs support this, do you know? > > > > > > Yes. As do f2fs, ocfs2, gfs2, ceph and NFSv4.2 > > > > Er... Nominally, gfs2 supports it. By treating all files as "there's a

Re: [PATCH 4/4] mm: migrate: Batch TLB flushing when unmapping pages for migration

2015-04-16 Thread Hugh Dickins
On Thu, 16 Apr 2015, Mel Gorman wrote: > Page reclaim batches multiple TLB flushes into one IPI and this patch teaches > page migration to also batch any necessary flushes. MMtests has a THP scale > microbenchmark that deliberately fragments memory and then allocates THPs > to stress compaction.

Re: [PATCH 2/4] mm: Send a single IPI to TLB flush multiple pages when unmapping

2015-04-16 Thread Hugh Dickins
On Thu, 16 Apr 2015, Mel Gorman wrote: > > /* Move the dirty bit to the physical page now the pte is gone. */ > - if (pte_dirty(pteval)) > + if (pte_dirty(pteval)) { > + /* > + * If the PTE was dirty then the TLB must be flushed before > + *

Re: [RFC 1/4] fs: Add generic file system event notifications

2015-04-16 Thread Hugh Dickins
On Thu, 16 Apr 2015, Beata Michalska wrote: > On 04/16/2015 05:46 AM, Eric Sandeen wrote: > > On 4/15/15 2:15 AM, Beata Michalska wrote: > >> Introduce configurable generic interface for file > >> system-wide event notifications to provide file > >> systems with a common way of reporting any

Re: [PATCHv3 4/5] mm: make compound_head() robust

2015-08-26 Thread Hugh Dickins
On Tue, 25 Aug 2015, Paul E. McKenney wrote: > On Tue, Aug 25, 2015 at 10:46:44PM +0200, Vlastimil Babka wrote: > > On 25.8.2015 22:11, Paul E. McKenney wrote: > > > On Tue, Aug 25, 2015 at 09:33:54PM +0300, Kirill A. Shutemov wrote: > > >> On Tue, Aug 25, 2015 at 01:44:13PM +0200, Vlastimil Babka

Re: [PATCHv3 4/5] mm: make compound_head() robust

2015-08-26 Thread Hugh Dickins
On Wed, 26 Aug 2015, Paul E. McKenney wrote: > On Wed, Aug 26, 2015 at 11:18:45AM -0700, Hugh Dickins wrote: > > On Tue, 25 Aug 2015, Paul E. McKenney wrote: > > > On Tue, Aug 25, 2015 at 10:46:44PM +0200, Vlastimil Babka wrote: > > > > On 25.8.2015 22:11, Paul E. Mc

Re: [PATCHv3 4/5] mm: make compound_head() robust

2015-08-27 Thread Hugh Dickins
On Thu, 27 Aug 2015, Michal Hocko wrote: > On Thu 27-08-15 17:09:17, Michal Hocko wrote: > [...] > > Btw. Do we need the same think for page::mapping and KSM? > > I guess we are safe here because the address for mappings comes from > kmalloc and that aligned properly, right? Not quite right, in

Re: [PATCH 2/2] mm: make compound_head() robust

2015-08-16 Thread Hugh Dickins
t;compound_head means that the page is tail. If the bit > set, rest of the page->compound_head is pointer to head page. Otherwise, > the field is NULL or pointer to memory cgroup. > > page->mem_cgroup currenly only used for small or head pages, so there > shouldn't be any conf

Re: [PATCHv2 3/4] mm: pack compound_dtor and compound_order into one word in struct page

2015-08-17 Thread Hugh Dickins
On Mon, 17 Aug 2015, Kirill A. Shutemov wrote: > The patch halves space occupied by compound_dtor and compound_order in > struct page. > > For compound_order, it's trivial long -> int/short conversion. > > For get_compound_page_dtor(), we now use hardcoded table for destructor > lookup and

Re: Potential data race in SyS_swapon

2015-08-17 Thread Hugh Dickins
On Fri, 7 Aug 2015, Andrey Konovalov wrote: > > We are working on a dynamic data race detector for the Linux kernel > called KernelThreadSanitizer (ktsan) > (https://github.com/google/ktsan/wiki). > > While running ktsan on the upstream revision 21bdb584af8c with trinity > we got a few reports

Re: [PATCH] shmem: recalculate file inode when fstat

2015-08-17 Thread Hugh Dickins
On Fri, 10 Jul 2015, Yu Zhao wrote: > Shmem uses shmem_recalc_inode to update i_blocks when it allocates > page, undoes range or swaps. But mm can drop clean page without > notifying shmem. This makes fstat sometimes return out-of-date > block size. > > The problem can be partially solved when

[PATCH] shmem: recalculate file inode when fstat

2015-08-17 Thread Hugh Dickins
ned triggers. Signed-off-by: Yu Zhao Signed-off-by: Hugh Dickins --- mm/shmem.c | 16 1 file changed, 16 insertions(+) diff --git a/mm/shmem.c b/mm/shmem.c index 4caf8ed..37e7933 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -542,6 +542,21 @@ void shmem_truncate_range(struct inode *in

[PATCH] mm: fix potential data race in SyS_swapon

2015-08-17 Thread Hugh Dickins
s_swapon"; and we should not replace appropriate -EBUSY by inappropriate -EINVAL. Index i was reused in a cpu loop further down: renamed cpu there. Reported-by: Andrey Konovalov Signed-off-by: Hugh Dickins --- mm/swapfile.c | 25 +++-- 1 file changed, 7 insertions(+), 18

Re: Multiple potential races on vma->vm_flags

2015-09-11 Thread Hugh Dickins
On Fri, 11 Sep 2015, Kirill A. Shutemov wrote: > On Thu, Sep 10, 2015 at 03:27:59PM +0200, Andrey Konovalov wrote: > > Can a vma be shared among a few mm's? > > Define "shared". > > vma can belong only to one process (mm_struct), but it can be accessed > from other process like in rmap case

Re: [PATCH] mm, vmscan: Do not wait for page writeback for GFP_NOFS allocations

2015-08-12 Thread Hugh Dickins
On Fri, 7 Aug 2015, Nikolay Borisov wrote: > On 08/04/2015 09:32 AM, Hugh Dickins wrote: > > > > And I've done quite a bit of testing. The loads that hung at the > > weekend have been running nicely for 24 hours now, no problem with the > > writeback hang and no prob

Re: page-flags behavior on compound pages: a worry

2015-08-12 Thread Hugh Dickins
On Thu, 13 Aug 2015, Kirill A. Shutemov wrote: > > All this situation is ugly. I'm thinking on more general solution for > PageTail() vs. ->first_page race. > > We would be able to avoid the race in first place if we encode PageTail() > and position of head page within the same word in struct

Re: page-flags behavior on compound pages: a worry

2015-08-12 Thread Hugh Dickins
On Fri, 7 Aug 2015, Kirill A. Shutemov wrote: > On Thu, Aug 06, 2015 at 12:24:22PM -0700, Hugh Dickins wrote: > > > > Oh, and I know a patchset which avoids these problems completely, > > by not using compound pages at all ;) > > BTW, I haven't heard anything abo

Re: [PATCHv3 0/5] Fix compound_head() race

2015-08-22 Thread Hugh Dickins
On Thu, 20 Aug 2015, Andrew Morton wrote: > On Thu, 20 Aug 2015 15:31:07 +0300 "Kirill A. Shutemov" > wrote: > > > On Wed, Aug 19, 2015 at 12:21:41PM +0300, Kirill A. Shutemov wrote: > > > Here's my attempt on fixing recently discovered race in compound_head(). > > > It should make

Re: mm: shmem_zero_setup skip security check and lockdep conflict with XFS

2015-07-10 Thread Hugh Dickins
On Thu, 9 Jul 2015, Stephen Smalley wrote: > On 07/09/2015 04:23 AM, Hugh Dickins wrote: > > On Wed, 8 Jul 2015, Stephen Smalley wrote: > >> On 07/08/2015 09:13 AM, Stephen Smalley wrote: > >>> On Sun, Jun 14, 2015 at 12:48 PM, Hugh Dickins wrote: > >>>

Re: [PATCH] selinux: fix mprotect PROT_EXEC regression caused by mm change

2015-07-11 Thread Hugh Dickins
k in that case. This makes the mmap and mprotect checking consistent > for shared anonymous mappings, as well as for /dev/zero and ashmem. > > Signed-off-by: Stephen Smalley Thank you for correcting that, Stephen (and for the nicely detailed commit description): it looks right to me so I'll

Re: [PATCH 3.12 78/78] mm: let mm_find_pmd fix buggy race with THP fault

2015-01-12 Thread Hugh Dickins
On Mon, 12 Jan 2015, Kirill A. Shutemov wrote: > On Mon, Jan 12, 2015 at 11:01:46AM +0100, Jiri Slaby wrote: > > On 01/10/2015, 06:01 AM, Hugh Dickins wrote: > > > On Fri, 9 Jan 2015, Jiri Slaby wrote: > > > > > >> From: Hugh Dickins > > >>

Re: [PATCH 3.12 78/78] mm: let mm_find_pmd fix buggy race with THP fault

2015-01-09 Thread Hugh Dickins
On Fri, 9 Jan 2015, Jiri Slaby wrote: > From: Hugh Dickins > > 3.12-stable review patch. If anyone has any objections, please let me know. > > === > > commit f72e7dcdd25229446b102e587ef2f826f76bff28 upstream. > > Trinity has reported: > >

Re: [patch 45/99] mm: unmapped page migration avoid unmap+remap overhead

2014-12-13 Thread Hugh Dickins
On Sat, 13 Dec 2014, Davidlohr Bueso wrote: > On Fri, 2014-12-12 at 16:56 -0800, a...@linux-foundation.org wrote: > > From: Hugh Dickins > > Subject: mm: unmapped page migration avoid unmap+remap overhead > > > > Page migration's __unmap_and_move(), and rmap's try_t

Re: [PATCH 08/10] mm/mremap: share the i_mmap_rwsem

2014-11-03 Thread Hugh Dickins
I'm glad to see this series back, and nicely presented: thank you. Not worth respinning them, but consider 1,2,3,4,5,6,7 and 9 as Acked-by: Hugh Dickins On Thu, 30 Oct 2014, Davidlohr Bueso wrote: > As per the comment in move_ptes(), we only require taking the > anon vma and i_mmap

Re: [PATCH 10/10] mm/hugetlb: share the i_mmap_rwsem

2014-11-03 Thread Hugh Dickins
On Thu, 30 Oct 2014, Davidlohr Bueso wrote: > The i_mmap_rwsem protects shared pages against races > when doing the sharing and unsharing, ultimately > calling huge_pmd_share/unshare() for PMD pages -- > it also needs it to avoid races when populating the pud > for pmd allocation when looking for

Re: [PATCH 10/10] mm/hugetlb: share the i_mmap_rwsem

2014-11-05 Thread Hugh Dickins
pages (ultimately via zap_page_range_single) without > touching the actual interval tree, thus share the lock. > > Signed-off-by: Davidlohr Bueso Acked-by: Hugh Dickins Yes, thanks, let's get this 11/10 into mmotm along with the rest, but put the hugetlb 10/10 on the shelf for now, until we've had time

Re: pipe/page fault oddness.

2014-10-01 Thread Hugh Dickins
ed, and he touched some of this > code last ("tag, you're it"). > > Kirill: the thread is on lkml, but basically it boils down to the > second byte write in fault_in_pages_writeable() faulting forever, > despite handle_mm_fault() apparently thinking that everythi

Re: pipe/page fault oddness.

2014-10-02 Thread Hugh Dickins
On Wed, 1 Oct 2014, Linus Torvalds wrote: > On Wed, Oct 1, 2014 at 1:19 AM, Hugh Dickins wrote: > > Can we please just get rid of _PAGE_NUMA. There is no excuse for it. I'm no lover of _PAGE_NUMA, and hope that it can be simplified away as you outline. What we have in 3.16+3.17 i

Re: [PATCH 0/5] mm: poison critical mm/ structs

2014-10-02 Thread Hugh Dickins
On Wed, 1 Oct 2014, Sasha Levin wrote: > On 10/01/2014 05:07 PM, Andrew Morton wrote: > > On Mon, 29 Sep 2014 21:47:14 -0400 Sasha Levin > > wrote: > > > >> Currently we're seeing a few issues which are unexplainable by looking at > >> the > >> data we see and are most likely caused by a

Re: [PATCH 1/2] mm: migrate: Close race between migration completion and mprotect

2014-10-02 Thread Hugh Dickins
double checking the VMA permissions using maybe_mkwrite when migration > completes. > > [torva...@linux-foundation.org: use maybe_mkwrite] > Cc: sta...@vger.kernel.org > Signed-off-by: Mel Gorman > Acked-by: Rik van Riel Sort-of-Acked-by: Hugh Dickins Safe patch, but I stand

Re: [PATCH 2/2] mm: numa: Do not mark PTEs pte_numa when splitting huge pages

2014-10-02 Thread Hugh Dickins
aults > will not be triggered which is marginal in comparison to the complexity > in dealing with the corner cases during THP split. > > Cc: sta...@vger.kernel.org > Signed-off-by: Mel Gorman > Acked-by: Rik van Riel > Acked-by: Kirill A. Shutemov Acked-by: Hugh Dickins except for wher

Re: [PATCH] mm: mempolicy: Skip inaccessible VMAs when setting MPOL_MF_LAZY

2014-10-02 Thread Hugh Dickins
userspace for now") but it should still skip VMAs the > same way task_numa_work does. > > Signed-off-by: Mel Gorman > Acked-by: Rik van Riel Acked-by: Hugh Dickins Yes, this is much the same as the patch I wrote for Linus two days ago, then discovered that we don't need un

Re: linux 4.1-rc7 deadlock

2015-06-11 Thread Hugh Dickins
s on kernel-internal inodes: the mmap("/dev/zero") cloned inode is indeed a kernel-internal detail. This also covers the !CONFIG_SHMEM use of ramfs to support /dev/zero (and MAP_SHARED|MAP_ANONYMOUS). I thought there were also drivers which cloned inode in mmap(), but if so, I cannot locate t

Re: linux 4.1-rc7 deadlock

2015-06-14 Thread Hugh Dickins
On Thu, 11 Jun 2015, Prarit Bhargava wrote: > On 06/11/2015 04:06 PM, Hugh Dickins wrote: > > On Tue, 9 Jun 2015, Morten Stevens wrote: > >> 2015-06-09 16:10 GMT+02:00 Daniel Wagner : > >>> On 06/09/2015 01:54 PM, Morten Stevens wrote: > > > > Reported-by

mm: shmem_zero_setup skip security check and lockdep conflict with XFS

2015-06-14 Thread Hugh Dickins
iel Wagner Reported-by: Morten Stevens Signed-off-by: Hugh Dickins --- mm/shmem.c |8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) --- 4.1-rc7/mm/shmem.c 2015-04-26 19:16:31.352191298 -0700 +++ linux/mm/shmem.c2015-06-14 09:26:49.461120166 -0700 @@ -3401,7 +3401,13 @@ int shm

<    2   3   4   5   6   7   8   9   10   11   >