Re: [PATCH 1/2] selftests: khugepaged: fix the shmem collapse failure

[Dev Jain]

On 12/06/25 9:24 am, Baolin Wang wrote:

When running the khugepaged selftest for shmem (./khugepaged all:shmem),
I encountered the following test failures:
"
Run test: collapse_full (khugepaged:shmem)
Collapse multiple fully populated PTE table.... Fail
...
Run test: collapse_single_pte_entry (khugepaged:shmem)
Collapse PTE table with single PTE entry present.... Fail
...
Run test: collapse_full_of_compound (khugepaged:shmem)
Allocate huge page... OK
Split huge page leaving single PTE page table full of compound pages... OK
Collapse PTE table full of compound pages.... Fail
"

The reason for the failure is that, it will set MADV_NOHUGEPAGE to prevent
khugepaged from continuing to scan shmem VMA after khugepaged finishes
scanning in the wait_for_scan() function. Moreover, shmem requires a refault
to establish PMD mappings.

However, after commit 2b0f922323cc, PMD mappings are prevented if the VMA is
set with MADV_NOHUGEPAGE flag, so shmem cannot establish PMD mappings during
refault.

To fix this issue, we can set the MADV_NOHUGEPAGE flag after the shmem refault.
With this fix, the shmem test case passes.

Fixes: 2b0f922323cc ("mm: don't install PMD mappings when THPs are disabled by the 
hw/process/vma")
Signed-off-by: Baolin Wang <baolin.w...@linux.alibaba.com>
---
  tools/testing/selftests/mm/khugepaged.c | 3 +--
  1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/tools/testing/selftests/mm/khugepaged.c 
b/tools/testing/selftests/mm/khugepaged.c
index 8a4d34cce36b..d462f62d8116 100644
--- a/tools/testing/selftests/mm/khugepaged.c
+++ b/tools/testing/selftests/mm/khugepaged.c
@@ -561,8 +561,6 @@ static bool wait_for_scan(const char *msg, char *p, int 
nr_hpages,
                usleep(TICK);
        }

- madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);

-
        return timeout == -1;
  }

@@ -585,6 +583,7 @@ static void khugepaged_collapse(const char *msg, char *p, int nr_hpages,

        if (ops != &__anon_ops)
                ops->fault(p, 0, nr_hpages * hpage_pmd_size);

+ madvise(p, nr_hpages * hpage_pmd_size, MADV_NOHUGEPAGE);

        if (ops->check_huge(p, expect ? nr_hpages : 0))
                success("OK");
        else

The idea looks sane to me, but do we need to add the madvise call to
madvise_retracted_page_tables() too, since that also calls wait_for_scan()?