[PATCH v2 13/18] mm/compaction: support non-lru movable page migration
On Wed, Mar 23, 2016 at 02:05:11PM +0900, Joonsoo Kim wrote: > On Tue, Mar 22, 2016 at 11:55:45PM +0900, Minchan Kim wrote: > > On Tue, Mar 22, 2016 at 02:50:37PM +0900, Joonsoo Kim wrote: > > > On Mon, Mar 21, 2016 at 03:31:02PM +0900, Minchan Kim wrote: > > > > We have allowed migration for only LRU pages until now and it was > > > > enough to make high-order pages. But recently, embedded system(e.g., > > > > webOS, android) uses lots of non-movable pages(e.g., zram, GPU memory) > > > > so we have seen several reports about troubles of small high-order > > > > allocation. For fixing the problem, there were several efforts > > > > (e,g,. enhance compaction algorithm, SLUB fallback to 0-order page, > > > > reserved memory, vmalloc and so on) but if there are lots of > > > > non-movable pages in system, their solutions are void in the long run. > > > > > > > > So, this patch is to support facility to change non-movable pages > > > > with movable. For the feature, this patch introduces functions related > > > > to migration to address_space_operations as well as some page flags. > > > > > > > > Basically, this patch supports two page-flags and two functions related > > > > to page migration. The flag and page->mapping stability are protected > > > > by PG_lock. > > > > > > > > PG_movable > > > > PG_isolated > > > > > > > > bool (*isolate_page) (struct page *, isolate_mode_t); > > > > void (*putback_page) (struct page *); > > > > > > > > Duty of subsystem want to make their pages as migratable are > > > > as follows: > > > > > > > > 1. It should register address_space to page->mapping then mark > > > > the page as PG_movable via __SetPageMovable. > > > > > > > > 2. It should mark the page as PG_isolated via SetPageIsolated > > > > if isolation is sucessful and return true. > > > > > > > > 3. If migration is successful, it should clear PG_isolated and > > > > PG_movable of the page for free preparation then release the > > > > reference of the page to free. > > > > > > > > 4. If migration fails, putback function of subsystem should > > > > clear PG_isolated via ClearPageIsolated. > > > > > > I think that this feature needs a separate document to describe > > > requirement of each step in more detail. For example, #1 can be > > > possible without holding a lock? I'm not sure because you lock > > > the page when implementing zsmalloc page migration in 15th patch. > > > > Yes, we needs PG_lock because install page->mapping and PG_movable > > should be atomic and PG_lock protects it. > > > > Better interface might be > > > > void __SetPageMovable(struct page *page, sruct address_space *mapping); > > > > > > > > #3 also need more explanation. Before release, we need to > > > unregister address_space. I guess that it needs to be done > > > in migratepage() but there is no explanation. > > > > Okay, we can unregister address_space in __ClearPageMovable. > > I will change it. > > > > > > > > > > > > > Cc: Vlastimil Babka > > > > Cc: Mel Gorman > > > > Cc: Hugh Dickins > > > > Cc: dri-devel at lists.freedesktop.org > > > > Cc: virtualization at lists.linux-foundation.org > > > > Signed-off-by: Gioh Kim > > > > Signed-off-by: Minchan Kim > > > > --- > > > > Documentation/filesystems/Locking | 4 + > > > > Documentation/filesystems/vfs.txt | 5 ++ > > > > fs/proc/page.c | 3 + > > > > include/linux/fs.h | 2 + > > > > include/linux/migrate.h| 2 + > > > > include/linux/page-flags.h | 29 > > > > include/uapi/linux/kernel-page-flags.h | 1 + > > > > mm/compaction.c| 14 +++- > > > > mm/migrate.c | 132 > > > > + > > > > 9 files changed, 177 insertions(+), 15 deletions(-) > > > > > > > > diff --git a/Documentation/filesystems/Locking > > > > b/Documentation/filesystems/Locking > > > > index 619af9bfdcb3..0bb79560abb3 100644 > > > > --- a/Documentation/filesystems/Locking > > > > +++ b/Documentation/filesystems/Locking > > > > @@ -195,7 +195,9 @@ unlocks and drops the reference. > > > > int (*releasepage) (struct page *, int); > > > > void (*freepage)(struct page *); > > > > int (*direct_IO)(struct kiocb *, struct iov_iter *iter, loff_t > > > > offset); > > > > + bool (*isolate_page) (struct page *, isolate_mode_t); > > > > int (*migratepage)(struct address_space *, struct page *, > > > > struct page *); > > > > + void (*putback_page) (struct page *); > > > > int (*launder_page)(struct page *); > > > > int (*is_partially_uptodate)(struct page *, unsigned long, > > > > unsigned long); > > > > int (*error_remove_page)(struct address_space *, struct page *); > > > > @@ -219,7 +221,9 @@ invalidatepage: yes > > > > releasepage: yes > > > > freepage: yes > > > > direct_IO: > > > >
[PATCH v2 13/18] mm/compaction: support non-lru movable page migration
On Tue, Mar 22, 2016 at 11:55:45PM +0900, Minchan Kim wrote: > On Tue, Mar 22, 2016 at 02:50:37PM +0900, Joonsoo Kim wrote: > > On Mon, Mar 21, 2016 at 03:31:02PM +0900, Minchan Kim wrote: > > > We have allowed migration for only LRU pages until now and it was > > > enough to make high-order pages. But recently, embedded system(e.g., > > > webOS, android) uses lots of non-movable pages(e.g., zram, GPU memory) > > > so we have seen several reports about troubles of small high-order > > > allocation. For fixing the problem, there were several efforts > > > (e,g,. enhance compaction algorithm, SLUB fallback to 0-order page, > > > reserved memory, vmalloc and so on) but if there are lots of > > > non-movable pages in system, their solutions are void in the long run. > > > > > > So, this patch is to support facility to change non-movable pages > > > with movable. For the feature, this patch introduces functions related > > > to migration to address_space_operations as well as some page flags. > > > > > > Basically, this patch supports two page-flags and two functions related > > > to page migration. The flag and page->mapping stability are protected > > > by PG_lock. > > > > > > PG_movable > > > PG_isolated > > > > > > bool (*isolate_page) (struct page *, isolate_mode_t); > > > void (*putback_page) (struct page *); > > > > > > Duty of subsystem want to make their pages as migratable are > > > as follows: > > > > > > 1. It should register address_space to page->mapping then mark > > > the page as PG_movable via __SetPageMovable. > > > > > > 2. It should mark the page as PG_isolated via SetPageIsolated > > > if isolation is sucessful and return true. > > > > > > 3. If migration is successful, it should clear PG_isolated and > > > PG_movable of the page for free preparation then release the > > > reference of the page to free. > > > > > > 4. If migration fails, putback function of subsystem should > > > clear PG_isolated via ClearPageIsolated. > > > > I think that this feature needs a separate document to describe > > requirement of each step in more detail. For example, #1 can be > > possible without holding a lock? I'm not sure because you lock > > the page when implementing zsmalloc page migration in 15th patch. > > Yes, we needs PG_lock because install page->mapping and PG_movable > should be atomic and PG_lock protects it. > > Better interface might be > > void __SetPageMovable(struct page *page, sruct address_space *mapping); > > > > > #3 also need more explanation. Before release, we need to > > unregister address_space. I guess that it needs to be done > > in migratepage() but there is no explanation. > > Okay, we can unregister address_space in __ClearPageMovable. > I will change it. > > > > > > > > > Cc: Vlastimil Babka > > > Cc: Mel Gorman > > > Cc: Hugh Dickins > > > Cc: dri-devel at lists.freedesktop.org > > > Cc: virtualization at lists.linux-foundation.org > > > Signed-off-by: Gioh Kim > > > Signed-off-by: Minchan Kim > > > --- > > > Documentation/filesystems/Locking | 4 + > > > Documentation/filesystems/vfs.txt | 5 ++ > > > fs/proc/page.c | 3 + > > > include/linux/fs.h | 2 + > > > include/linux/migrate.h| 2 + > > > include/linux/page-flags.h | 29 > > > include/uapi/linux/kernel-page-flags.h | 1 + > > > mm/compaction.c| 14 +++- > > > mm/migrate.c | 132 > > > + > > > 9 files changed, 177 insertions(+), 15 deletions(-) > > > > > > diff --git a/Documentation/filesystems/Locking > > > b/Documentation/filesystems/Locking > > > index 619af9bfdcb3..0bb79560abb3 100644 > > > --- a/Documentation/filesystems/Locking > > > +++ b/Documentation/filesystems/Locking > > > @@ -195,7 +195,9 @@ unlocks and drops the reference. > > > int (*releasepage) (struct page *, int); > > > void (*freepage)(struct page *); > > > int (*direct_IO)(struct kiocb *, struct iov_iter *iter, loff_t offset); > > > + bool (*isolate_page) (struct page *, isolate_mode_t); > > > int (*migratepage)(struct address_space *, struct page *, struct page > > > *); > > > + void (*putback_page) (struct page *); > > > int (*launder_page)(struct page *); > > > int (*is_partially_uptodate)(struct page *, unsigned long, unsigned > > > long); > > > int (*error_remove_page)(struct address_space *, struct page *); > > > @@ -219,7 +221,9 @@ invalidatepage: yes > > > releasepage: yes > > > freepage:yes > > > direct_IO: > > > +isolate_page:yes > > > migratepage: yes (both) > > > +putback_page:yes > > > launder_page:yes > > > is_partially_uptodate: yes > > > error_remove_page: yes > > > diff --git a/Documentation/filesystems/vfs.txt > > > b/Documentation/filesystems/vfs.txt > > > index b02a7d598258..4c1b6c3b4bc8 1
[PATCH v2 13/18] mm/compaction: support non-lru movable page migration
On Tue, Mar 22, 2016 at 02:50:37PM +0900, Joonsoo Kim wrote: > On Mon, Mar 21, 2016 at 03:31:02PM +0900, Minchan Kim wrote: > > We have allowed migration for only LRU pages until now and it was > > enough to make high-order pages. But recently, embedded system(e.g., > > webOS, android) uses lots of non-movable pages(e.g., zram, GPU memory) > > so we have seen several reports about troubles of small high-order > > allocation. For fixing the problem, there were several efforts > > (e,g,. enhance compaction algorithm, SLUB fallback to 0-order page, > > reserved memory, vmalloc and so on) but if there are lots of > > non-movable pages in system, their solutions are void in the long run. > > > > So, this patch is to support facility to change non-movable pages > > with movable. For the feature, this patch introduces functions related > > to migration to address_space_operations as well as some page flags. > > > > Basically, this patch supports two page-flags and two functions related > > to page migration. The flag and page->mapping stability are protected > > by PG_lock. > > > > PG_movable > > PG_isolated > > > > bool (*isolate_page) (struct page *, isolate_mode_t); > > void (*putback_page) (struct page *); > > > > Duty of subsystem want to make their pages as migratable are > > as follows: > > > > 1. It should register address_space to page->mapping then mark > > the page as PG_movable via __SetPageMovable. > > > > 2. It should mark the page as PG_isolated via SetPageIsolated > > if isolation is sucessful and return true. > > > > 3. If migration is successful, it should clear PG_isolated and > > PG_movable of the page for free preparation then release the > > reference of the page to free. > > > > 4. If migration fails, putback function of subsystem should > > clear PG_isolated via ClearPageIsolated. > > I think that this feature needs a separate document to describe > requirement of each step in more detail. For example, #1 can be > possible without holding a lock? I'm not sure because you lock > the page when implementing zsmalloc page migration in 15th patch. Yes, we needs PG_lock because install page->mapping and PG_movable should be atomic and PG_lock protects it. Better interface might be void __SetPageMovable(struct page *page, sruct address_space *mapping); > > #3 also need more explanation. Before release, we need to > unregister address_space. I guess that it needs to be done > in migratepage() but there is no explanation. Okay, we can unregister address_space in __ClearPageMovable. I will change it. > > > > > Cc: Vlastimil Babka > > Cc: Mel Gorman > > Cc: Hugh Dickins > > Cc: dri-devel at lists.freedesktop.org > > Cc: virtualization at lists.linux-foundation.org > > Signed-off-by: Gioh Kim > > Signed-off-by: Minchan Kim > > --- > > Documentation/filesystems/Locking | 4 + > > Documentation/filesystems/vfs.txt | 5 ++ > > fs/proc/page.c | 3 + > > include/linux/fs.h | 2 + > > include/linux/migrate.h| 2 + > > include/linux/page-flags.h | 29 > > include/uapi/linux/kernel-page-flags.h | 1 + > > mm/compaction.c| 14 +++- > > mm/migrate.c | 132 > > + > > 9 files changed, 177 insertions(+), 15 deletions(-) > > > > diff --git a/Documentation/filesystems/Locking > > b/Documentation/filesystems/Locking > > index 619af9bfdcb3..0bb79560abb3 100644 > > --- a/Documentation/filesystems/Locking > > +++ b/Documentation/filesystems/Locking > > @@ -195,7 +195,9 @@ unlocks and drops the reference. > > int (*releasepage) (struct page *, int); > > void (*freepage)(struct page *); > > int (*direct_IO)(struct kiocb *, struct iov_iter *iter, loff_t offset); > > + bool (*isolate_page) (struct page *, isolate_mode_t); > > int (*migratepage)(struct address_space *, struct page *, struct page > > *); > > + void (*putback_page) (struct page *); > > int (*launder_page)(struct page *); > > int (*is_partially_uptodate)(struct page *, unsigned long, unsigned > > long); > > int (*error_remove_page)(struct address_space *, struct page *); > > @@ -219,7 +221,9 @@ invalidatepage: yes > > releasepage: yes > > freepage: yes > > direct_IO: > > +isolate_page: yes > > migratepage: yes (both) > > +putback_page: yes > > launder_page: yes > > is_partially_uptodate: yes > > error_remove_page: yes > > diff --git a/Documentation/filesystems/vfs.txt > > b/Documentation/filesystems/vfs.txt > > index b02a7d598258..4c1b6c3b4bc8 100644 > > --- a/Documentation/filesystems/vfs.txt > > +++ b/Documentation/filesystems/vfs.txt > > @@ -592,9 +592,14 @@ struct address_space_operations { > > int (*releasepage) (struct page *, int); > > void (*freepage)(struct page *); > > ssize_t (*direct
[PATCH v2 13/18] mm/compaction: support non-lru movable page migration
On Mon, Mar 21, 2016 at 03:31:02PM +0900, Minchan Kim wrote: > We have allowed migration for only LRU pages until now and it was > enough to make high-order pages. But recently, embedded system(e.g., > webOS, android) uses lots of non-movable pages(e.g., zram, GPU memory) > so we have seen several reports about troubles of small high-order > allocation. For fixing the problem, there were several efforts > (e,g,. enhance compaction algorithm, SLUB fallback to 0-order page, > reserved memory, vmalloc and so on) but if there are lots of > non-movable pages in system, their solutions are void in the long run. > > So, this patch is to support facility to change non-movable pages > with movable. For the feature, this patch introduces functions related > to migration to address_space_operations as well as some page flags. > > Basically, this patch supports two page-flags and two functions related > to page migration. The flag and page->mapping stability are protected > by PG_lock. > > PG_movable > PG_isolated > > bool (*isolate_page) (struct page *, isolate_mode_t); > void (*putback_page) (struct page *); > > Duty of subsystem want to make their pages as migratable are > as follows: > > 1. It should register address_space to page->mapping then mark > the page as PG_movable via __SetPageMovable. > > 2. It should mark the page as PG_isolated via SetPageIsolated > if isolation is sucessful and return true. > > 3. If migration is successful, it should clear PG_isolated and > PG_movable of the page for free preparation then release the > reference of the page to free. > > 4. If migration fails, putback function of subsystem should > clear PG_isolated via ClearPageIsolated. I think that this feature needs a separate document to describe requirement of each step in more detail. For example, #1 can be possible without holding a lock? I'm not sure because you lock the page when implementing zsmalloc page migration in 15th patch. #3 also need more explanation. Before release, we need to unregister address_space. I guess that it needs to be done in migratepage() but there is no explanation. > > Cc: Vlastimil Babka > Cc: Mel Gorman > Cc: Hugh Dickins > Cc: dri-devel at lists.freedesktop.org > Cc: virtualization at lists.linux-foundation.org > Signed-off-by: Gioh Kim > Signed-off-by: Minchan Kim > --- > Documentation/filesystems/Locking | 4 + > Documentation/filesystems/vfs.txt | 5 ++ > fs/proc/page.c | 3 + > include/linux/fs.h | 2 + > include/linux/migrate.h| 2 + > include/linux/page-flags.h | 29 > include/uapi/linux/kernel-page-flags.h | 1 + > mm/compaction.c| 14 +++- > mm/migrate.c | 132 > + > 9 files changed, 177 insertions(+), 15 deletions(-) > > diff --git a/Documentation/filesystems/Locking > b/Documentation/filesystems/Locking > index 619af9bfdcb3..0bb79560abb3 100644 > --- a/Documentation/filesystems/Locking > +++ b/Documentation/filesystems/Locking > @@ -195,7 +195,9 @@ unlocks and drops the reference. > int (*releasepage) (struct page *, int); > void (*freepage)(struct page *); > int (*direct_IO)(struct kiocb *, struct iov_iter *iter, loff_t offset); > + bool (*isolate_page) (struct page *, isolate_mode_t); > int (*migratepage)(struct address_space *, struct page *, struct page > *); > + void (*putback_page) (struct page *); > int (*launder_page)(struct page *); > int (*is_partially_uptodate)(struct page *, unsigned long, unsigned > long); > int (*error_remove_page)(struct address_space *, struct page *); > @@ -219,7 +221,9 @@ invalidatepage: yes > releasepage: yes > freepage:yes > direct_IO: > +isolate_page:yes > migratepage: yes (both) > +putback_page:yes > launder_page:yes > is_partially_uptodate: yes > error_remove_page: yes > diff --git a/Documentation/filesystems/vfs.txt > b/Documentation/filesystems/vfs.txt > index b02a7d598258..4c1b6c3b4bc8 100644 > --- a/Documentation/filesystems/vfs.txt > +++ b/Documentation/filesystems/vfs.txt > @@ -592,9 +592,14 @@ struct address_space_operations { > int (*releasepage) (struct page *, int); > void (*freepage)(struct page *); > ssize_t (*direct_IO)(struct kiocb *, struct iov_iter *iter, loff_t > offset); > + /* isolate a page for migration */ > + bool (*isolate_page) (struct page *, isolate_mode_t); > /* migrate the contents of a page to the specified target */ > int (*migratepage) (struct page *, struct page *); > + /* put the page back to right list */ > + void (*putback_page) (struct page *); > int (*launder_page) (struct page *); > + > int (*is_partially_uptodate) (struct page *, unsigned long, >
[PATCH v2 13/18] mm/compaction: support non-lru movable page migration
We have allowed migration for only LRU pages until now and it was enough to make high-order pages. But recently, embedded system(e.g., webOS, android) uses lots of non-movable pages(e.g., zram, GPU memory) so we have seen several reports about troubles of small high-order allocation. For fixing the problem, there were several efforts (e,g,. enhance compaction algorithm, SLUB fallback to 0-order page, reserved memory, vmalloc and so on) but if there are lots of non-movable pages in system, their solutions are void in the long run. So, this patch is to support facility to change non-movable pages with movable. For the feature, this patch introduces functions related to migration to address_space_operations as well as some page flags. Basically, this patch supports two page-flags and two functions related to page migration. The flag and page->mapping stability are protected by PG_lock. PG_movable PG_isolated bool (*isolate_page) (struct page *, isolate_mode_t); void (*putback_page) (struct page *); Duty of subsystem want to make their pages as migratable are as follows: 1. It should register address_space to page->mapping then mark the page as PG_movable via __SetPageMovable. 2. It should mark the page as PG_isolated via SetPageIsolated if isolation is sucessful and return true. 3. If migration is successful, it should clear PG_isolated and PG_movable of the page for free preparation then release the reference of the page to free. 4. If migration fails, putback function of subsystem should clear PG_isolated via ClearPageIsolated. Cc: Vlastimil Babka Cc: Mel Gorman Cc: Hugh Dickins Cc: dri-devel at lists.freedesktop.org Cc: virtualization at lists.linux-foundation.org Signed-off-by: Gioh Kim Signed-off-by: Minchan Kim --- Documentation/filesystems/Locking | 4 + Documentation/filesystems/vfs.txt | 5 ++ fs/proc/page.c | 3 + include/linux/fs.h | 2 + include/linux/migrate.h| 2 + include/linux/page-flags.h | 29 include/uapi/linux/kernel-page-flags.h | 1 + mm/compaction.c| 14 +++- mm/migrate.c | 132 + 9 files changed, 177 insertions(+), 15 deletions(-) diff --git a/Documentation/filesystems/Locking b/Documentation/filesystems/Locking index 619af9bfdcb3..0bb79560abb3 100644 --- a/Documentation/filesystems/Locking +++ b/Documentation/filesystems/Locking @@ -195,7 +195,9 @@ unlocks and drops the reference. int (*releasepage) (struct page *, int); void (*freepage)(struct page *); int (*direct_IO)(struct kiocb *, struct iov_iter *iter, loff_t offset); + bool (*isolate_page) (struct page *, isolate_mode_t); int (*migratepage)(struct address_space *, struct page *, struct page *); + void (*putback_page) (struct page *); int (*launder_page)(struct page *); int (*is_partially_uptodate)(struct page *, unsigned long, unsigned long); int (*error_remove_page)(struct address_space *, struct page *); @@ -219,7 +221,9 @@ invalidatepage: yes releasepage: yes freepage: yes direct_IO: +isolate_page: yes migratepage: yes (both) +putback_page: yes launder_page: yes is_partially_uptodate: yes error_remove_page: yes diff --git a/Documentation/filesystems/vfs.txt b/Documentation/filesystems/vfs.txt index b02a7d598258..4c1b6c3b4bc8 100644 --- a/Documentation/filesystems/vfs.txt +++ b/Documentation/filesystems/vfs.txt @@ -592,9 +592,14 @@ struct address_space_operations { int (*releasepage) (struct page *, int); void (*freepage)(struct page *); ssize_t (*direct_IO)(struct kiocb *, struct iov_iter *iter, loff_t offset); + /* isolate a page for migration */ + bool (*isolate_page) (struct page *, isolate_mode_t); /* migrate the contents of a page to the specified target */ int (*migratepage) (struct page *, struct page *); + /* put the page back to right list */ + void (*putback_page) (struct page *); int (*launder_page) (struct page *); + int (*is_partially_uptodate) (struct page *, unsigned long, unsigned long); void (*is_dirty_writeback) (struct page *, bool *, bool *); diff --git a/fs/proc/page.c b/fs/proc/page.c index 712f1b9992cc..e2066e73a9b8 100644 --- a/fs/proc/page.c +++ b/fs/proc/page.c @@ -157,6 +157,9 @@ u64 stable_page_flags(struct page *page) if (page_is_idle(page)) u |= 1 << KPF_IDLE; + if (PageMovable(page)) + u |= 1 << KPF_MOVABLE; + u |= kpf_copy_bit(k, KPF_LOCKED,PG_locked); u |= kpf_copy_bit(k, KPF_SLAB, PG_slab); diff --git a/include/linux/fs.h b/include/linux/fs.h index 14a97194b34b..b7ef2e41fa4a 100644 --- a/include/linux/fs