[patch] mm, thp: do not cause memcg oom for thp

[David Rientjes]

Commit 2516035499b9 ("mm, thp: remove __GFP_NORETRY from khugepaged and
madvised allocations") changed the page allocator to no longer detect thp
allocations based on __GFP_NORETRY.

It did not, however, modify the mem cgroup try_charge() path to avoid oom
kill for either khugepaged collapsing or thp faulting.  It is never
expected to oom kill a process to allocate a hugepage for thp; reclaim is
governed by the thp defrag mode and MADV_HUGEPAGE, but allocations (and
charging) should fallback instead of oom killing processes.

Fixes: 2516035499b9 ("mm, thp: remove __GFP_NORETRY from khugepaged and 
madvised allocations")
Signed-off-by: David Rientjes <rient...@google.com>
---
 mm/huge_memory.c | 5 +++--
 mm/khugepaged.c  | 8 ++++++--
 2 files changed, 9 insertions(+), 4 deletions(-)

diff --git a/mm/huge_memory.c b/mm/huge_memory.c
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -555,7 +555,8 @@ static int __do_huge_pmd_anonymous_page(struct vm_fault 
*vmf, struct page *page,
 
        VM_BUG_ON_PAGE(!PageCompound(page), page);
 
-       if (mem_cgroup_try_charge(page, vma->vm_mm, gfp, &memcg, true)) {
+       if (mem_cgroup_try_charge(page, vma->vm_mm, gfp | __GFP_NORETRY, &memcg,
+                                 true)) {
                put_page(page);
                count_vm_event(THP_FAULT_FALLBACK);
                return VM_FAULT_FALLBACK;
@@ -1316,7 +1317,7 @@ int do_huge_pmd_wp_page(struct vm_fault *vmf, pmd_t 
orig_pmd)
        }
 
        if (unlikely(mem_cgroup_try_charge(new_page, vma->vm_mm,
-                                       huge_gfp, &memcg, true))) {
+                               huge_gfp | __GFP_NORETRY, &memcg, true))) {
                put_page(new_page);
                split_huge_pmd(vma, vmf->pmd, vmf->address);
                if (page)
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -960,7 +960,9 @@ static void collapse_huge_page(struct mm_struct *mm,
                goto out_nolock;
        }
 
-       if (unlikely(mem_cgroup_try_charge(new_page, mm, gfp, &memcg, true))) {
+       /* Do not oom kill for khugepaged charges */
+       if (unlikely(mem_cgroup_try_charge(new_page, mm, gfp | __GFP_NORETRY,
+                                          &memcg, true))) {
                result = SCAN_CGROUP_CHARGE_FAIL;
                goto out_nolock;
        }
@@ -1319,7 +1321,9 @@ static void collapse_shmem(struct mm_struct *mm,
                goto out;
        }
 
-       if (unlikely(mem_cgroup_try_charge(new_page, mm, gfp, &memcg, true))) {
+       /* Do not oom kill for khugepaged charges */
+       if (unlikely(mem_cgroup_try_charge(new_page, mm, gfp | __GFP_NORETRY,
+                                          &memcg, true))) {
                result = SCAN_CGROUP_CHARGE_FAIL;
                goto out;
        }