Re: [PATCH 05/10] mm/migrate: demote pages during reclaim
> +static unsigned int demote_page_list(struct list_head *demote_pages, > +struct pglist_data *pgdat, > +struct scan_control *sc) sc is not needed and can be removed from demote_page_list(). Reviewed-by: Wei Xu
Re: [PATCH 05/10] mm/migrate: demote pages during reclaim
On Thu, Apr 01, 2021 at 11:32:25AM -0700, Dave Hansen wrote: > > From: Dave Hansen > > This is mostly derived from a patch from Yang Shi: > > > https://lore.kernel.org/linux-mm/1560468577-101178-10-git-send-email-yang@linux.alibaba.com/ > > Add code to the reclaim path (shrink_page_list()) to "demote" data > to another NUMA node instead of discarding the data. This always > avoids the cost of I/O needed to read the page back in and sometimes > avoids the writeout cost when the pagee is dirty. > > A second pass through shrink_page_list() will be made if any demotions > fail. This essentally falls back to normal reclaim behavior in the > case that demotions fail. Previous versions of this patch may have > simply failed to reclaim pages which were eligible for demotion but > were unable to be demoted in practice. > > Note: This just adds the start of infratructure for migration. It is > actually disabled next to the FIXME in migrate_demote_page_ok(). > > Signed-off-by: Dave Hansen > Cc: Wei Xu > Cc: Yang Shi > Cc: David Rientjes > Cc: Huang Ying > Cc: Dan Williams > Cc: osalvador Reviewed-by: Oscar Salvador -- Oscar Salvador SUSE L3
Re: [PATCH 05/10] mm/migrate: demote pages during reclaim
On 4/1/21 1:01 PM, Yang Shi wrote: > On Thu, Apr 1, 2021 at 11:35 AM Dave Hansen > wrote: >> >> >> From: Dave Hansen >> >> This is mostly derived from a patch from Yang Shi: >> >> >> https://lore.kernel.org/linux-mm/1560468577-101178-10-git-send-email-yang@linux.alibaba.com/ >> >> Add code to the reclaim path (shrink_page_list()) to "demote" data >> to another NUMA node instead of discarding the data. This always >> avoids the cost of I/O needed to read the page back in and sometimes >> avoids the writeout cost when the pagee is dirty. > > s/pagee/page > >> >> A second pass through shrink_page_list() will be made if any demotions >> fail. This essentally falls back to normal reclaim behavior in the > > s/essentally/essentially > >> case that demotions fail. Previous versions of this patch may have >> simply failed to reclaim pages which were eligible for demotion but >> were unable to be demoted in practice. >> >> Note: This just adds the start of infratructure for migration. It is > > s/infratructure/infrastructure Thanks for finding those! I somehow got really sloppy in this patch. I scanned the rest of the descriptions and don't see any more obvious typos/misspellings. >> +static struct page *alloc_demote_page(struct page *page, unsigned long node) >> +{ >> + struct migration_target_control mtc = { >> + /* >> +* Allocate from 'node', or fail the quickly and quietly. >> +* When this happens, 'page; will likely just be discarded > > s/'page;/'page' Fixed. Thanks for the review tag!
Re: [PATCH 05/10] mm/migrate: demote pages during reclaim
On Thu, Apr 1, 2021 at 11:35 AM Dave Hansen wrote: > > > From: Dave Hansen > > This is mostly derived from a patch from Yang Shi: > > > https://lore.kernel.org/linux-mm/1560468577-101178-10-git-send-email-yang@linux.alibaba.com/ > > Add code to the reclaim path (shrink_page_list()) to "demote" data > to another NUMA node instead of discarding the data. This always > avoids the cost of I/O needed to read the page back in and sometimes > avoids the writeout cost when the pagee is dirty. s/pagee/page > > A second pass through shrink_page_list() will be made if any demotions > fail. This essentally falls back to normal reclaim behavior in the s/essentally/essentially > case that demotions fail. Previous versions of this patch may have > simply failed to reclaim pages which were eligible for demotion but > were unable to be demoted in practice. > > Note: This just adds the start of infratructure for migration. It is s/infratructure/infrastructure > actually disabled next to the FIXME in migrate_demote_page_ok(). > > Signed-off-by: Dave Hansen > Cc: Wei Xu > Cc: Yang Shi > Cc: David Rientjes > Cc: Huang Ying > Cc: Dan Williams > Cc: osalvador > > -- > changes from 20210122: > * move from GFP_HIGHUSER -> GFP_HIGHUSER_MOVABLE (Ying) > > changes from 202010: > * add MR_NUMA_MISPLACED to trace MIGRATE_REASON define > * make migrate_demote_page_ok() static, remove 'sc' arg until >later patch > * remove unnecessary alloc_demote_page() hugetlb warning > * Simplify alloc_demote_page() gfp mask. Depend on >__GFP_NORETRY to make it lightweight instead of fancier >stuff like leaving out __GFP_IO/FS. > * Allocate migration page with alloc_migration_target() >instead of allocating directly. > changes from 20200730: > * Add another pass through shrink_page_list() when demotion >fails. > changes from 20210302: > * Use __GFP_THISNODE and revise the comment explaining the >GFP mask constructionn Other than some typos above this patch looks good to me. Reviewed-by: Yang Shi Another nit below: > --- > > b/include/linux/migrate.h|9 > b/include/trace/events/migrate.h |3 - > b/mm/vmscan.c| 82 > +++ > 3 files changed, 93 insertions(+), 1 deletion(-) > > diff -puN include/linux/migrate.h~demote-with-migrate_pages > include/linux/migrate.h > --- a/include/linux/migrate.h~demote-with-migrate_pages 2021-03-31 > 15:17:15.842000251 -0700 > +++ b/include/linux/migrate.h 2021-03-31 15:17:15.853000251 -0700 > @@ -27,6 +27,7 @@ enum migrate_reason { > MR_MEMPOLICY_MBIND, > MR_NUMA_MISPLACED, > MR_CONTIG_RANGE, > + MR_DEMOTION, > MR_TYPES > }; > > @@ -196,6 +197,14 @@ struct migrate_vma { > int migrate_vma_setup(struct migrate_vma *args); > void migrate_vma_pages(struct migrate_vma *migrate); > void migrate_vma_finalize(struct migrate_vma *migrate); > +int next_demotion_node(int node); > + > +#else /* CONFIG_MIGRATION disabled: */ > + > +static inline int next_demotion_node(int node) > +{ > + return NUMA_NO_NODE; > +} > > #endif /* CONFIG_MIGRATION */ > > diff -puN include/trace/events/migrate.h~demote-with-migrate_pages > include/trace/events/migrate.h > --- a/include/trace/events/migrate.h~demote-with-migrate_pages 2021-03-31 > 15:17:15.846000251 -0700 > +++ b/include/trace/events/migrate.h2021-03-31 15:17:15.853000251 -0700 > @@ -20,7 +20,8 @@ > EM( MR_SYSCALL, "syscall_or_cpuset")\ > EM( MR_MEMPOLICY_MBIND, "mempolicy_mbind") \ > EM( MR_NUMA_MISPLACED, "numa_misplaced") \ > - EMe(MR_CONTIG_RANGE,"contig_range") > + EM( MR_CONTIG_RANGE,"contig_range") \ > + EMe(MR_DEMOTION,"demotion") > > /* > * First define the enums in the above macros to be exported to userspace > diff -puN mm/vmscan.c~demote-with-migrate_pages mm/vmscan.c > --- a/mm/vmscan.c~demote-with-migrate_pages 2021-03-31 15:17:15.848000251 > -0700 > +++ b/mm/vmscan.c 2021-03-31 15:17:15.856000251 -0700 > @@ -41,6 +41,7 @@ > #include > #include > #include > +#include > #include > #include > #include > @@ -1035,6 +1036,23 @@ static enum page_references page_check_r > return PAGEREF_RECLAIM; > } > > +static bool migrate_demote_page_ok(struct page *page) > +{ > + int next_nid = next_demotion_node(page_to_nid(page)); > + > + VM_BUG_ON_PAGE(!PageLocked(page), page); > + VM_BUG_ON_PAGE(PageHuge(page), page); > + VM_BUG_ON_PAGE(PageLRU(page), page); > + > + if (next_nid == NUMA_NO_NODE) > + return false; > + if (PageTransHuge(page) && !thp_migration_supported()) > + return false; > + > + // FIXME: actually enable this later in the series > + return false; > +} > + > /* Check if a page is dirty or under writeback */ > static void
[PATCH 05/10] mm/migrate: demote pages during reclaim
From: Dave Hansen This is mostly derived from a patch from Yang Shi: https://lore.kernel.org/linux-mm/1560468577-101178-10-git-send-email-yang@linux.alibaba.com/ Add code to the reclaim path (shrink_page_list()) to "demote" data to another NUMA node instead of discarding the data. This always avoids the cost of I/O needed to read the page back in and sometimes avoids the writeout cost when the pagee is dirty. A second pass through shrink_page_list() will be made if any demotions fail. This essentally falls back to normal reclaim behavior in the case that demotions fail. Previous versions of this patch may have simply failed to reclaim pages which were eligible for demotion but were unable to be demoted in practice. Note: This just adds the start of infratructure for migration. It is actually disabled next to the FIXME in migrate_demote_page_ok(). Signed-off-by: Dave Hansen Cc: Wei Xu Cc: Yang Shi Cc: David Rientjes Cc: Huang Ying Cc: Dan Williams Cc: osalvador -- changes from 20210122: * move from GFP_HIGHUSER -> GFP_HIGHUSER_MOVABLE (Ying) changes from 202010: * add MR_NUMA_MISPLACED to trace MIGRATE_REASON define * make migrate_demote_page_ok() static, remove 'sc' arg until later patch * remove unnecessary alloc_demote_page() hugetlb warning * Simplify alloc_demote_page() gfp mask. Depend on __GFP_NORETRY to make it lightweight instead of fancier stuff like leaving out __GFP_IO/FS. * Allocate migration page with alloc_migration_target() instead of allocating directly. changes from 20200730: * Add another pass through shrink_page_list() when demotion fails. changes from 20210302: * Use __GFP_THISNODE and revise the comment explaining the GFP mask constructionn --- b/include/linux/migrate.h|9 b/include/trace/events/migrate.h |3 - b/mm/vmscan.c| 82 +++ 3 files changed, 93 insertions(+), 1 deletion(-) diff -puN include/linux/migrate.h~demote-with-migrate_pages include/linux/migrate.h --- a/include/linux/migrate.h~demote-with-migrate_pages 2021-03-31 15:17:15.842000251 -0700 +++ b/include/linux/migrate.h 2021-03-31 15:17:15.853000251 -0700 @@ -27,6 +27,7 @@ enum migrate_reason { MR_MEMPOLICY_MBIND, MR_NUMA_MISPLACED, MR_CONTIG_RANGE, + MR_DEMOTION, MR_TYPES }; @@ -196,6 +197,14 @@ struct migrate_vma { int migrate_vma_setup(struct migrate_vma *args); void migrate_vma_pages(struct migrate_vma *migrate); void migrate_vma_finalize(struct migrate_vma *migrate); +int next_demotion_node(int node); + +#else /* CONFIG_MIGRATION disabled: */ + +static inline int next_demotion_node(int node) +{ + return NUMA_NO_NODE; +} #endif /* CONFIG_MIGRATION */ diff -puN include/trace/events/migrate.h~demote-with-migrate_pages include/trace/events/migrate.h --- a/include/trace/events/migrate.h~demote-with-migrate_pages 2021-03-31 15:17:15.846000251 -0700 +++ b/include/trace/events/migrate.h2021-03-31 15:17:15.853000251 -0700 @@ -20,7 +20,8 @@ EM( MR_SYSCALL, "syscall_or_cpuset")\ EM( MR_MEMPOLICY_MBIND, "mempolicy_mbind") \ EM( MR_NUMA_MISPLACED, "numa_misplaced") \ - EMe(MR_CONTIG_RANGE,"contig_range") + EM( MR_CONTIG_RANGE,"contig_range") \ + EMe(MR_DEMOTION,"demotion") /* * First define the enums in the above macros to be exported to userspace diff -puN mm/vmscan.c~demote-with-migrate_pages mm/vmscan.c --- a/mm/vmscan.c~demote-with-migrate_pages 2021-03-31 15:17:15.848000251 -0700 +++ b/mm/vmscan.c 2021-03-31 15:17:15.856000251 -0700 @@ -41,6 +41,7 @@ #include #include #include +#include #include #include #include @@ -1035,6 +1036,23 @@ static enum page_references page_check_r return PAGEREF_RECLAIM; } +static bool migrate_demote_page_ok(struct page *page) +{ + int next_nid = next_demotion_node(page_to_nid(page)); + + VM_BUG_ON_PAGE(!PageLocked(page), page); + VM_BUG_ON_PAGE(PageHuge(page), page); + VM_BUG_ON_PAGE(PageLRU(page), page); + + if (next_nid == NUMA_NO_NODE) + return false; + if (PageTransHuge(page) && !thp_migration_supported()) + return false; + + // FIXME: actually enable this later in the series + return false; +} + /* Check if a page is dirty or under writeback */ static void page_check_dirty_writeback(struct page *page, bool *dirty, bool *writeback) @@ -1065,6 +1083,46 @@ static void page_check_dirty_writeback(s mapping->a_ops->is_dirty_writeback(page, dirty, writeback); } +static struct page *alloc_demote_page(struct page *page, unsigned long node) +{ + struct migration_target_control mtc = { + /* +* Allocate from 'node', or fail the quickly and quietly. +*
Re: [PATCH 05/10] mm/migrate: demote pages during reclaim
On 3/8/21 4:10 PM, Yang Shi wrote: >> +static struct page *alloc_demote_page(struct page *page, unsigned long node) >> +{ >> + struct migration_target_control mtc = { >> + /* >> +* Fail the allocation quickly and quietly. When this >> +* happens, 'page; will likely just be discarded instead >> +* of migrated. >> +*/ >> + .gfp_mask = GFP_HIGHUSER_MOVABLE | __GFP_NORETRY | >> __GFP_NOWARN, >> + .nid = node > I recall I pointed out __GFP_THISNODE should be needed to guarantee > the allocation doesn't fallback. But it seems it is not solved yet or > it is guaranteed by the other way? Sorry about missing that before. This mask definitely needs __GFP_THISNODE. I've added it for the next version.
Re: [PATCH 05/10] mm/migrate: demote pages during reclaim
On Thu, Mar 4, 2021 at 4:01 PM Dave Hansen wrote: > > > From: Dave Hansen > > This is mostly derived from a patch from Yang Shi: > > > https://lore.kernel.org/linux-mm/1560468577-101178-10-git-send-email-yang@linux.alibaba.com/ > > Add code to the reclaim path (shrink_page_list()) to "demote" data > to another NUMA node instead of discarding the data. This always > avoids the cost of I/O needed to read the page back in and sometimes > avoids the writeout cost when the pagee is dirty. > > A second pass through shrink_page_list() will be made if any demotions > fail. This essentally falls back to normal reclaim behavior in the > case that demotions fail. Previous versions of this patch may have > simply failed to reclaim pages which were eligible for demotion but > were unable to be demoted in practice. > > Note: This just adds the start of infratructure for migration. It is > actually disabled next to the FIXME in migrate_demote_page_ok(). > > Signed-off-by: Dave Hansen > Cc: Yang Shi > Cc: David Rientjes > Cc: Huang Ying > Cc: Dan Williams > Cc: osalvador > > -- > changes from 20210122: > * move from GFP_HIGHUSER -> GFP_HIGHUSER_MOVABLE (Ying) > > changes from 202010: > * add MR_NUMA_MISPLACED to trace MIGRATE_REASON define > * make migrate_demote_page_ok() static, remove 'sc' arg until >later patch > * remove unnecessary alloc_demote_page() hugetlb warning > * Simplify alloc_demote_page() gfp mask. Depend on >__GFP_NORETRY to make it lightweight instead of fancier >stuff like leaving out __GFP_IO/FS. > * Allocate migration page with alloc_migration_target() >instead of allocating directly. > changes from 20200730: > * Add another pass through shrink_page_list() when demotion >fails. > --- > > b/include/linux/migrate.h| 13 +- > b/include/trace/events/migrate.h |3 - > b/mm/vmscan.c| 81 > +++ > 3 files changed, 94 insertions(+), 3 deletions(-) > > diff -puN include/linux/migrate.h~demote-with-migrate_pages > include/linux/migrate.h > --- a/include/linux/migrate.h~demote-with-migrate_pages 2021-03-04 > 15:35:56.471806429 -0800 > +++ b/include/linux/migrate.h 2021-03-04 15:35:56.479806429 -0800 > @@ -27,6 +27,7 @@ enum migrate_reason { > MR_MEMPOLICY_MBIND, > MR_NUMA_MISPLACED, > MR_CONTIG_RANGE, > + MR_DEMOTION, > MR_TYPES > }; > > @@ -58,8 +59,8 @@ extern int migrate_page_move_mapping(str > > static inline void putback_movable_pages(struct list_head *l) {} > static inline int migrate_pages(struct list_head *l, new_page_t new, > - unsigned long private, enum migrate_mode mode, int reason, > - unsigned int *nr_succeeded) > + free_page_t free, unsigned long private, enum migrate_mode > mode, > + int reason, unsigned int *nr_succeeded) > { return -ENOSYS; } > static inline struct page *alloc_migration_target(struct page *page, > unsigned long private) > @@ -196,6 +197,14 @@ struct migrate_vma { > int migrate_vma_setup(struct migrate_vma *args); > void migrate_vma_pages(struct migrate_vma *migrate); > void migrate_vma_finalize(struct migrate_vma *migrate); > +int next_demotion_node(int node); > + > +#else /* CONFIG_MIGRATION disabled: */ > + > +static inline int next_demotion_node(int node) > +{ > + return NUMA_NO_NODE; > +} > > #endif /* CONFIG_MIGRATION */ > > diff -puN include/trace/events/migrate.h~demote-with-migrate_pages > include/trace/events/migrate.h > --- a/include/trace/events/migrate.h~demote-with-migrate_pages 2021-03-04 > 15:35:56.473806429 -0800 > +++ b/include/trace/events/migrate.h2021-03-04 15:35:56.479806429 -0800 > @@ -20,7 +20,8 @@ > EM( MR_SYSCALL, "syscall_or_cpuset")\ > EM( MR_MEMPOLICY_MBIND, "mempolicy_mbind") \ > EM( MR_NUMA_MISPLACED, "numa_misplaced") \ > - EMe(MR_CONTIG_RANGE,"contig_range") > + EM( MR_CONTIG_RANGE,"contig_range") \ > + EMe(MR_DEMOTION,"demotion") > > /* > * First define the enums in the above macros to be exported to userspace > diff -puN mm/vmscan.c~demote-with-migrate_pages mm/vmscan.c > --- a/mm/vmscan.c~demote-with-migrate_pages 2021-03-04 15:35:56.475806429 > -0800 > +++ b/mm/vmscan.c 2021-03-04 15:35:56.482806429 -0800 > @@ -41,6 +41,7 @@ > #include > #include > #include > +#include > #include > #include > #include > @@ -1034,6 +1035,23 @@ static enum page_references page_check_r > return PAGEREF_RECLAIM; > } > > +static bool migrate_demote_page_ok(struct page *page) > +{ > + int next_nid = next_demotion_node(page_to_nid(page)); > + > + VM_BUG_ON_PAGE(!PageLocked(page), page); > + VM_BUG_ON_PAGE(PageHuge(page), page); > + VM_BUG_ON_PAGE(PageLRU(page), page); > + > + if (next_nid == NUMA_NO_NODE) > +
[PATCH 05/10] mm/migrate: demote pages during reclaim
From: Dave Hansen This is mostly derived from a patch from Yang Shi: https://lore.kernel.org/linux-mm/1560468577-101178-10-git-send-email-yang@linux.alibaba.com/ Add code to the reclaim path (shrink_page_list()) to "demote" data to another NUMA node instead of discarding the data. This always avoids the cost of I/O needed to read the page back in and sometimes avoids the writeout cost when the pagee is dirty. A second pass through shrink_page_list() will be made if any demotions fail. This essentally falls back to normal reclaim behavior in the case that demotions fail. Previous versions of this patch may have simply failed to reclaim pages which were eligible for demotion but were unable to be demoted in practice. Note: This just adds the start of infratructure for migration. It is actually disabled next to the FIXME in migrate_demote_page_ok(). Signed-off-by: Dave Hansen Cc: Yang Shi Cc: David Rientjes Cc: Huang Ying Cc: Dan Williams Cc: osalvador -- changes from 20210122: * move from GFP_HIGHUSER -> GFP_HIGHUSER_MOVABLE (Ying) changes from 202010: * add MR_NUMA_MISPLACED to trace MIGRATE_REASON define * make migrate_demote_page_ok() static, remove 'sc' arg until later patch * remove unnecessary alloc_demote_page() hugetlb warning * Simplify alloc_demote_page() gfp mask. Depend on __GFP_NORETRY to make it lightweight instead of fancier stuff like leaving out __GFP_IO/FS. * Allocate migration page with alloc_migration_target() instead of allocating directly. changes from 20200730: * Add another pass through shrink_page_list() when demotion fails. --- b/include/linux/migrate.h| 13 +- b/include/trace/events/migrate.h |3 - b/mm/vmscan.c| 81 +++ 3 files changed, 94 insertions(+), 3 deletions(-) diff -puN include/linux/migrate.h~demote-with-migrate_pages include/linux/migrate.h --- a/include/linux/migrate.h~demote-with-migrate_pages 2021-03-04 15:35:56.471806429 -0800 +++ b/include/linux/migrate.h 2021-03-04 15:35:56.479806429 -0800 @@ -27,6 +27,7 @@ enum migrate_reason { MR_MEMPOLICY_MBIND, MR_NUMA_MISPLACED, MR_CONTIG_RANGE, + MR_DEMOTION, MR_TYPES }; @@ -58,8 +59,8 @@ extern int migrate_page_move_mapping(str static inline void putback_movable_pages(struct list_head *l) {} static inline int migrate_pages(struct list_head *l, new_page_t new, - unsigned long private, enum migrate_mode mode, int reason, - unsigned int *nr_succeeded) + free_page_t free, unsigned long private, enum migrate_mode mode, + int reason, unsigned int *nr_succeeded) { return -ENOSYS; } static inline struct page *alloc_migration_target(struct page *page, unsigned long private) @@ -196,6 +197,14 @@ struct migrate_vma { int migrate_vma_setup(struct migrate_vma *args); void migrate_vma_pages(struct migrate_vma *migrate); void migrate_vma_finalize(struct migrate_vma *migrate); +int next_demotion_node(int node); + +#else /* CONFIG_MIGRATION disabled: */ + +static inline int next_demotion_node(int node) +{ + return NUMA_NO_NODE; +} #endif /* CONFIG_MIGRATION */ diff -puN include/trace/events/migrate.h~demote-with-migrate_pages include/trace/events/migrate.h --- a/include/trace/events/migrate.h~demote-with-migrate_pages 2021-03-04 15:35:56.473806429 -0800 +++ b/include/trace/events/migrate.h2021-03-04 15:35:56.479806429 -0800 @@ -20,7 +20,8 @@ EM( MR_SYSCALL, "syscall_or_cpuset")\ EM( MR_MEMPOLICY_MBIND, "mempolicy_mbind") \ EM( MR_NUMA_MISPLACED, "numa_misplaced") \ - EMe(MR_CONTIG_RANGE,"contig_range") + EM( MR_CONTIG_RANGE,"contig_range") \ + EMe(MR_DEMOTION,"demotion") /* * First define the enums in the above macros to be exported to userspace diff -puN mm/vmscan.c~demote-with-migrate_pages mm/vmscan.c --- a/mm/vmscan.c~demote-with-migrate_pages 2021-03-04 15:35:56.475806429 -0800 +++ b/mm/vmscan.c 2021-03-04 15:35:56.482806429 -0800 @@ -41,6 +41,7 @@ #include #include #include +#include #include #include #include @@ -1034,6 +1035,23 @@ static enum page_references page_check_r return PAGEREF_RECLAIM; } +static bool migrate_demote_page_ok(struct page *page) +{ + int next_nid = next_demotion_node(page_to_nid(page)); + + VM_BUG_ON_PAGE(!PageLocked(page), page); + VM_BUG_ON_PAGE(PageHuge(page), page); + VM_BUG_ON_PAGE(PageLRU(page), page); + + if (next_nid == NUMA_NO_NODE) + return false; + if (PageTransHuge(page) && !thp_migration_supported()) + return false; + + // FIXME: actually enable this later in the series + return false; +} + /* Check if a page is dirty or under writeback */ static void