[PATCH v2 13/18] mm/compaction: support non-lru movable page migration
On Wed, Mar 23, 2016 at 02:05:11PM +0900, Joonsoo Kim wrote: > On Tue, Mar 22, 2016 at 11:55:45PM +0900, Minchan Kim wrote: > > On Tue, Mar 22, 2016 at 02:50:37PM +0900, Joonsoo Kim wrote: > > > On Mon, Mar 21, 2016 at 03:31:02PM +0900, Minchan Kim wrote: > > > > We have allowed migration for only LRU pages until now and it was > > > > enough to make high-order pages. But recently, embedded system(e.g., > > > > webOS, android) uses lots of non-movable pages(e.g., zram, GPU memory) > > > > so we have seen several reports about troubles of small high-order > > > > allocation. For fixing the problem, there were several efforts > > > > (e,g,. enhance compaction algorithm, SLUB fallback to 0-order page, > > > > reserved memory, vmalloc and so on) but if there are lots of > > > > non-movable pages in system, their solutions are void in the long run. > > > > > > > > So, this patch is to support facility to change non-movable pages > > > > with movable. For the feature, this patch introduces functions related > > > > to migration to address_space_operations as well as some page flags. > > > > > > > > Basically, this patch supports two page-flags and two functions related > > > > to page migration. The flag and page->mapping stability are protected > > > > by PG_lock. > > > > > > > > PG_movable > > > > PG_isolated > > > > > > > > bool (*isolate_page) (struct page *, isolate_mode_t); > > > > void (*putback_page) (struct page *); > > > > > > > > Duty of subsystem want to make their pages as migratable are > > > > as follows: > > > > > > > > 1. It should register address_space to page->mapping then mark > > > > the page as PG_movable via __SetPageMovable. > > > > > > > > 2. It should mark the page as PG_isolated via SetPageIsolated > > > > if isolation is sucessful and return true. > > > > > > > > 3. If migration is successful, it should clear PG_isolated and > > > > PG_movable of the page for free preparation then release the > > > > reference of the page to free. > > > > > > > > 4. If migration fails, putback function of subsystem should > > > > clear PG_isolated via ClearPageIsolated. > > > > > > I think that this feature needs a separate document to describe > > > requirement of each step in more detail. For example, #1 can be > > > possible without holding a lock? I'm not sure because you lock > > > the page when implementing zsmalloc page migration in 15th patch. > > > > Yes, we needs PG_lock because install page->mapping and PG_movable > > should be atomic and PG_lock protects it. > > > > Better interface might be > > > > void __SetPageMovable(struct page *page, sruct address_space *mapping); > > > > > > > > #3 also need more explanation. Before release, we need to > > > unregister address_space. I guess that it needs to be done > > > in migratepage() but there is no explanation. > > > > Okay, we can unregister address_space in __ClearPageMovable. > > I will change it. > > > > > > > > > > > > > Cc: Vlastimil Babka > > > > Cc: Mel Gorman > > > > Cc: Hugh Dickins > > > > Cc: dri-devel at lists.freedesktop.org > > > > Cc: virtualization at lists.linux-foundation.org > > > > Signed-off-by: Gioh Kim > > > > Signed-off-by: Minchan Kim > > > > --- > > > > Documentation/filesystems/Locking | 4 + > > > > Documentation/filesystems/vfs.txt | 5 ++ > > > > fs/proc/page.c | 3 + > > > > include/linux/fs.h | 2 + > > > > include/linux/migrate.h| 2 + > > > > include/linux/page-flags.h | 29 > > > > include/uapi/linux/kernel-page-flags.h | 1 + > > > > mm/compaction.c| 14 +++- > > > > mm/migrate.c | 132 > > > > + > > > > 9 files changed, 177 insertions(+), 15 deletions(-) > > > > > > > > diff --git a/Documentation/filesystems/Locking > > > > b/Documentation/filesystems/Locking > > > > index 619af9bfdcb3..0bb79560abb3 100644 > > > > --- a/Documentation/filesystems/Locking > > > > +++ b/Documentation/filesystems/Locking > > > > @@ -195,7 +195,9 @@ unlocks and drops the reference. > > > > int (*releasepage) (struct page *, int); > > > > void (*freepage)(struct page *); > > > > int (*direct_IO)(struct kiocb *, struct iov_iter *iter, loff_t > > > > offset); > > > > + bool (*isolate_page) (struct page *, isolate_mode_t); > > > > int (*migratepage)(struct address_space *, struct page *, > > > > struct page *); > > > > + void (*putback_page) (struct page *); > > > > int (*launder_page)(struct page *); > > > > int (*is_partially_uptodate)(struct page *, unsigned long, > > > > unsigned long); > > > > int (*error_remove_page)(struct address_space *, struct page *); > > > > @@ -219,7 +221,9 @@ invalidatepage: yes > > > > releasepage: yes > > > > freepage: yes > > > > direct_IO: > > >
[PATCH v2 13/18] mm/compaction: support non-lru movable page migration
On Tue, Mar 22, 2016 at 11:55:45PM +0900, Minchan Kim wrote: > On Tue, Mar 22, 2016 at 02:50:37PM +0900, Joonsoo Kim wrote: > > On Mon, Mar 21, 2016 at 03:31:02PM +0900, Minchan Kim wrote: > > > We have allowed migration for only LRU pages until now and it was > > > enough to make high-order pages. But recently, embedded system(e.g., > > > webOS, android) uses lots of non-movable pages(e.g., zram, GPU memory) > > > so we have seen several reports about troubles of small high-order > > > allocation. For fixing the problem, there were several efforts > > > (e,g,. enhance compaction algorithm, SLUB fallback to 0-order page, > > > reserved memory, vmalloc and so on) but if there are lots of > > > non-movable pages in system, their solutions are void in the long run. > > > > > > So, this patch is to support facility to change non-movable pages > > > with movable. For the feature, this patch introduces functions related > > > to migration to address_space_operations as well as some page flags. > > > > > > Basically, this patch supports two page-flags and two functions related > > > to page migration. The flag and page->mapping stability are protected > > > by PG_lock. > > > > > > PG_movable > > > PG_isolated > > > > > > bool (*isolate_page) (struct page *, isolate_mode_t); > > > void (*putback_page) (struct page *); > > > > > > Duty of subsystem want to make their pages as migratable are > > > as follows: > > > > > > 1. It should register address_space to page->mapping then mark > > > the page as PG_movable via __SetPageMovable. > > > > > > 2. It should mark the page as PG_isolated via SetPageIsolated > > > if isolation is sucessful and return true. > > > > > > 3. If migration is successful, it should clear PG_isolated and > > > PG_movable of the page for free preparation then release the > > > reference of the page to free. > > > > > > 4. If migration fails, putback function of subsystem should > > > clear PG_isolated via ClearPageIsolated. > > > > I think that this feature needs a separate document to describe > > requirement of each step in more detail. For example, #1 can be > > possible without holding a lock? I'm not sure because you lock > > the page when implementing zsmalloc page migration in 15th patch. > > Yes, we needs PG_lock because install page->mapping and PG_movable > should be atomic and PG_lock protects it. > > Better interface might be > > void __SetPageMovable(struct page *page, sruct address_space *mapping); > > > > > #3 also need more explanation. Before release, we need to > > unregister address_space. I guess that it needs to be done > > in migratepage() but there is no explanation. > > Okay, we can unregister address_space in __ClearPageMovable. > I will change it. > > > > > > > > > Cc: Vlastimil Babka > > > Cc: Mel Gorman > > > Cc: Hugh Dickins > > > Cc: dri-devel at lists.freedesktop.org > > > Cc: virtualization at lists.linux-foundation.org > > > Signed-off-by: Gioh Kim > > > Signed-off-by: Minchan Kim > > > --- > > > Documentation/filesystems/Locking | 4 + > > > Documentation/filesystems/vfs.txt | 5 ++ > > > fs/proc/page.c | 3 + > > > include/linux/fs.h | 2 + > > > include/linux/migrate.h| 2 + > > > include/linux/page-flags.h | 29 > > > include/uapi/linux/kernel-page-flags.h | 1 + > > > mm/compaction.c| 14 +++- > > > mm/migrate.c | 132 > > > + > > > 9 files changed, 177 insertions(+), 15 deletions(-) > > > > > > diff --git a/Documentation/filesystems/Locking > > > b/Documentation/filesystems/Locking > > > index 619af9bfdcb3..0bb79560abb3 100644 > > > --- a/Documentation/filesystems/Locking > > > +++ b/Documentation/filesystems/Locking > > > @@ -195,7 +195,9 @@ unlocks and drops the reference. > > > int (*releasepage) (struct page *, int); > > > void (*freepage)(struct page *); > > > int (*direct_IO)(struct kiocb *, struct iov_iter *iter, loff_t offset); > > > + bool (*isolate_page) (struct page *, isolate_mode_t); > > > int (*migratepage)(struct address_space *, struct page *, struct page > > > *); > > > + void (*putback_page) (struct page *); > > > int (*launder_page)(struct page *); > > > int (*is_partially_uptodate)(struct page *, unsigned long, unsigned > > > long); > > > int (*error_remove_page)(struct address_space *, struct page *); > > > @@ -219,7 +221,9 @@ invalidatepage: yes > > > releasepage: yes > > > freepage:yes > > > direct_IO: > > > +isolate_page:yes > > > migratepage: yes (both) > > > +putback_page:yes > > > launder_page:yes > > > is_partially_uptodate: yes > > > error_remove_page: yes > > > diff --git a/Documentation/filesystems/vfs.txt > > > b/Documentation/filesystems/vfs.txt > > > index b02a7d598258..4c1b6c3b4bc8
[PATCH v2 13/18] mm/compaction: support non-lru movable page migration
On Tue, Mar 22, 2016 at 02:50:37PM +0900, Joonsoo Kim wrote:
> On Mon, Mar 21, 2016 at 03:31:02PM +0900, Minchan Kim wrote:
> > We have allowed migration for only LRU pages until now and it was
> > enough to make high-order pages. But recently, embedded system(e.g.,
> > webOS, android) uses lots of non-movable pages(e.g., zram, GPU memory)
> > so we have seen several reports about troubles of small high-order
> > allocation. For fixing the problem, there were several efforts
> > (e,g,. enhance compaction algorithm, SLUB fallback to 0-order page,
> > reserved memory, vmalloc and so on) but if there are lots of
> > non-movable pages in system, their solutions are void in the long run.
> >
> > So, this patch is to support facility to change non-movable pages
> > with movable. For the feature, this patch introduces functions related
> > to migration to address_space_operations as well as some page flags.
> >
> > Basically, this patch supports two page-flags and two functions related
> > to page migration. The flag and page->mapping stability are protected
> > by PG_lock.
> >
> > PG_movable
> > PG_isolated
> >
> > bool (*isolate_page) (struct page *, isolate_mode_t);
> > void (*putback_page) (struct page *);
> >
> > Duty of subsystem want to make their pages as migratable are
> > as follows:
> >
> > 1. It should register address_space to page->mapping then mark
> > the page as PG_movable via __SetPageMovable.
> >
> > 2. It should mark the page as PG_isolated via SetPageIsolated
> > if isolation is sucessful and return true.
> >
> > 3. If migration is successful, it should clear PG_isolated and
> > PG_movable of the page for free preparation then release the
> > reference of the page to free.
> >
> > 4. If migration fails, putback function of subsystem should
> > clear PG_isolated via ClearPageIsolated.
>
> I think that this feature needs a separate document to describe
> requirement of each step in more detail. For example, #1 can be
> possible without holding a lock? I'm not sure because you lock
> the page when implementing zsmalloc page migration in 15th patch.
Yes, we needs PG_lock because install page->mapping and PG_movable
should be atomic and PG_lock protects it.
Better interface might be
void __SetPageMovable(struct page *page, sruct address_space *mapping);
>
> #3 also need more explanation. Before release, we need to
> unregister address_space. I guess that it needs to be done
> in migratepage() but there is no explanation.
Okay, we can unregister address_space in __ClearPageMovable.
I will change it.
>
> >
> > Cc: Vlastimil Babka
> > Cc: Mel Gorman
> > Cc: Hugh Dickins
> > Cc: dri-devel at lists.freedesktop.org
> > Cc: virtualization at lists.linux-foundation.org
> > Signed-off-by: Gioh Kim
> > Signed-off-by: Minchan Kim
> > ---
> > Documentation/filesystems/Locking | 4 +
> > Documentation/filesystems/vfs.txt | 5 ++
> > fs/proc/page.c | 3 +
> > include/linux/fs.h | 2 +
> > include/linux/migrate.h| 2 +
> > include/linux/page-flags.h | 29
> > include/uapi/linux/kernel-page-flags.h | 1 +
> > mm/compaction.c| 14 +++-
> > mm/migrate.c | 132
> > +
> > 9 files changed, 177 insertions(+), 15 deletions(-)
> >
> > diff --git a/Documentation/filesystems/Locking
> > b/Documentation/filesystems/Locking
> > index 619af9bfdcb3..0bb79560abb3 100644
> > --- a/Documentation/filesystems/Locking
> > +++ b/Documentation/filesystems/Locking
> > @@ -195,7 +195,9 @@ unlocks and drops the reference.
> > int (*releasepage) (struct page *, int);
> > void (*freepage)(struct page *);
> > int (*direct_IO)(struct kiocb *, struct iov_iter *iter, loff_t offset);
> > + bool (*isolate_page) (struct page *, isolate_mode_t);
> > int (*migratepage)(struct address_space *, struct page *, struct page
> > *);
> > + void (*putback_page) (struct page *);
> > int (*launder_page)(struct page *);
> > int (*is_partially_uptodate)(struct page *, unsigned long, unsigned
> > long);
> > int (*error_remove_page)(struct address_space *, struct page *);
> > @@ -219,7 +221,9 @@ invalidatepage: yes
> > releasepage: yes
> > freepage: yes
> > direct_IO:
> > +isolate_page: yes
> > migratepage: yes (both)
> > +putback_page: yes
> > launder_page: yes
> > is_partially_uptodate: yes
> > error_remove_page: yes
> > diff --git a/Documentation/filesystems/vfs.txt
> > b/Documentation/filesystems/vfs.txt
> > index b02a7d598258..4c1b6c3b4bc8 100644
> > --- a/Documentation/filesystems/vfs.txt
> > +++ b/Documentation/filesystems/vfs.txt
> > @@ -592,9 +592,14 @@ struct address_space_operations {
> > int (*releasepage) (struct page *, int);
> > void (*freepage)(struct page *);
> > ssize_t
[PATCH v2 13/18] mm/compaction: support non-lru movable page migration
On Mon, Mar 21, 2016 at 03:31:02PM +0900, Minchan Kim wrote:
> We have allowed migration for only LRU pages until now and it was
> enough to make high-order pages. But recently, embedded system(e.g.,
> webOS, android) uses lots of non-movable pages(e.g., zram, GPU memory)
> so we have seen several reports about troubles of small high-order
> allocation. For fixing the problem, there were several efforts
> (e,g,. enhance compaction algorithm, SLUB fallback to 0-order page,
> reserved memory, vmalloc and so on) but if there are lots of
> non-movable pages in system, their solutions are void in the long run.
>
> So, this patch is to support facility to change non-movable pages
> with movable. For the feature, this patch introduces functions related
> to migration to address_space_operations as well as some page flags.
>
> Basically, this patch supports two page-flags and two functions related
> to page migration. The flag and page->mapping stability are protected
> by PG_lock.
>
> PG_movable
> PG_isolated
>
> bool (*isolate_page) (struct page *, isolate_mode_t);
> void (*putback_page) (struct page *);
>
> Duty of subsystem want to make their pages as migratable are
> as follows:
>
> 1. It should register address_space to page->mapping then mark
> the page as PG_movable via __SetPageMovable.
>
> 2. It should mark the page as PG_isolated via SetPageIsolated
> if isolation is sucessful and return true.
>
> 3. If migration is successful, it should clear PG_isolated and
> PG_movable of the page for free preparation then release the
> reference of the page to free.
>
> 4. If migration fails, putback function of subsystem should
> clear PG_isolated via ClearPageIsolated.
I think that this feature needs a separate document to describe
requirement of each step in more detail. For example, #1 can be
possible without holding a lock? I'm not sure because you lock
the page when implementing zsmalloc page migration in 15th patch.
#3 also need more explanation. Before release, we need to
unregister address_space. I guess that it needs to be done
in migratepage() but there is no explanation.
>
> Cc: Vlastimil Babka
> Cc: Mel Gorman
> Cc: Hugh Dickins
> Cc: dri-devel at lists.freedesktop.org
> Cc: virtualization at lists.linux-foundation.org
> Signed-off-by: Gioh Kim
> Signed-off-by: Minchan Kim
> ---
> Documentation/filesystems/Locking | 4 +
> Documentation/filesystems/vfs.txt | 5 ++
> fs/proc/page.c | 3 +
> include/linux/fs.h | 2 +
> include/linux/migrate.h| 2 +
> include/linux/page-flags.h | 29
> include/uapi/linux/kernel-page-flags.h | 1 +
> mm/compaction.c| 14 +++-
> mm/migrate.c | 132
> +
> 9 files changed, 177 insertions(+), 15 deletions(-)
>
> diff --git a/Documentation/filesystems/Locking
> b/Documentation/filesystems/Locking
> index 619af9bfdcb3..0bb79560abb3 100644
> --- a/Documentation/filesystems/Locking
> +++ b/Documentation/filesystems/Locking
> @@ -195,7 +195,9 @@ unlocks and drops the reference.
> int (*releasepage) (struct page *, int);
> void (*freepage)(struct page *);
> int (*direct_IO)(struct kiocb *, struct iov_iter *iter, loff_t offset);
> + bool (*isolate_page) (struct page *, isolate_mode_t);
> int (*migratepage)(struct address_space *, struct page *, struct page
> *);
> + void (*putback_page) (struct page *);
> int (*launder_page)(struct page *);
> int (*is_partially_uptodate)(struct page *, unsigned long, unsigned
> long);
> int (*error_remove_page)(struct address_space *, struct page *);
> @@ -219,7 +221,9 @@ invalidatepage: yes
> releasepage: yes
> freepage:yes
> direct_IO:
> +isolate_page:yes
> migratepage: yes (both)
> +putback_page:yes
> launder_page:yes
> is_partially_uptodate: yes
> error_remove_page: yes
> diff --git a/Documentation/filesystems/vfs.txt
> b/Documentation/filesystems/vfs.txt
> index b02a7d598258..4c1b6c3b4bc8 100644
> --- a/Documentation/filesystems/vfs.txt
> +++ b/Documentation/filesystems/vfs.txt
> @@ -592,9 +592,14 @@ struct address_space_operations {
> int (*releasepage) (struct page *, int);
> void (*freepage)(struct page *);
> ssize_t (*direct_IO)(struct kiocb *, struct iov_iter *iter, loff_t
> offset);
> + /* isolate a page for migration */
> + bool (*isolate_page) (struct page *, isolate_mode_t);
> /* migrate the contents of a page to the specified target */
> int (*migratepage) (struct page *, struct page *);
> + /* put the page back to right list */
> + void (*putback_page) (struct page *);
> int (*launder_page) (struct page *);
> +
> int (*is_partially_uptodate) (struct page *, unsigned long,
>
[PATCH v2 13/18] mm/compaction: support non-lru movable page migration
We have allowed migration for only LRU pages until now and it was
enough to make high-order pages. But recently, embedded system(e.g.,
webOS, android) uses lots of non-movable pages(e.g., zram, GPU memory)
so we have seen several reports about troubles of small high-order
allocation. For fixing the problem, there were several efforts
(e,g,. enhance compaction algorithm, SLUB fallback to 0-order page,
reserved memory, vmalloc and so on) but if there are lots of
non-movable pages in system, their solutions are void in the long run.
So, this patch is to support facility to change non-movable pages
with movable. For the feature, this patch introduces functions related
to migration to address_space_operations as well as some page flags.
Basically, this patch supports two page-flags and two functions related
to page migration. The flag and page->mapping stability are protected
by PG_lock.
PG_movable
PG_isolated
bool (*isolate_page) (struct page *, isolate_mode_t);
void (*putback_page) (struct page *);
Duty of subsystem want to make their pages as migratable are
as follows:
1. It should register address_space to page->mapping then mark
the page as PG_movable via __SetPageMovable.
2. It should mark the page as PG_isolated via SetPageIsolated
if isolation is sucessful and return true.
3. If migration is successful, it should clear PG_isolated and
PG_movable of the page for free preparation then release the
reference of the page to free.
4. If migration fails, putback function of subsystem should
clear PG_isolated via ClearPageIsolated.
Cc: Vlastimil Babka
Cc: Mel Gorman
Cc: Hugh Dickins
Cc: dri-devel at lists.freedesktop.org
Cc: virtualization at lists.linux-foundation.org
Signed-off-by: Gioh Kim
Signed-off-by: Minchan Kim
---
Documentation/filesystems/Locking | 4 +
Documentation/filesystems/vfs.txt | 5 ++
fs/proc/page.c | 3 +
include/linux/fs.h | 2 +
include/linux/migrate.h| 2 +
include/linux/page-flags.h | 29
include/uapi/linux/kernel-page-flags.h | 1 +
mm/compaction.c| 14 +++-
mm/migrate.c | 132 +
9 files changed, 177 insertions(+), 15 deletions(-)
diff --git a/Documentation/filesystems/Locking
b/Documentation/filesystems/Locking
index 619af9bfdcb3..0bb79560abb3 100644
--- a/Documentation/filesystems/Locking
+++ b/Documentation/filesystems/Locking
@@ -195,7 +195,9 @@ unlocks and drops the reference.
int (*releasepage) (struct page *, int);
void (*freepage)(struct page *);
int (*direct_IO)(struct kiocb *, struct iov_iter *iter, loff_t offset);
+ bool (*isolate_page) (struct page *, isolate_mode_t);
int (*migratepage)(struct address_space *, struct page *, struct page
*);
+ void (*putback_page) (struct page *);
int (*launder_page)(struct page *);
int (*is_partially_uptodate)(struct page *, unsigned long, unsigned
long);
int (*error_remove_page)(struct address_space *, struct page *);
@@ -219,7 +221,9 @@ invalidatepage: yes
releasepage: yes
freepage: yes
direct_IO:
+isolate_page: yes
migratepage: yes (both)
+putback_page: yes
launder_page: yes
is_partially_uptodate: yes
error_remove_page: yes
diff --git a/Documentation/filesystems/vfs.txt
b/Documentation/filesystems/vfs.txt
index b02a7d598258..4c1b6c3b4bc8 100644
--- a/Documentation/filesystems/vfs.txt
+++ b/Documentation/filesystems/vfs.txt
@@ -592,9 +592,14 @@ struct address_space_operations {
int (*releasepage) (struct page *, int);
void (*freepage)(struct page *);
ssize_t (*direct_IO)(struct kiocb *, struct iov_iter *iter, loff_t
offset);
+ /* isolate a page for migration */
+ bool (*isolate_page) (struct page *, isolate_mode_t);
/* migrate the contents of a page to the specified target */
int (*migratepage) (struct page *, struct page *);
+ /* put the page back to right list */
+ void (*putback_page) (struct page *);
int (*launder_page) (struct page *);
+
int (*is_partially_uptodate) (struct page *, unsigned long,
unsigned long);
void (*is_dirty_writeback) (struct page *, bool *, bool *);
diff --git a/fs/proc/page.c b/fs/proc/page.c
index 712f1b9992cc..e2066e73a9b8 100644
--- a/fs/proc/page.c
+++ b/fs/proc/page.c
@@ -157,6 +157,9 @@ u64 stable_page_flags(struct page *page)
if (page_is_idle(page))
u |= 1 << KPF_IDLE;
+ if (PageMovable(page))
+ u |= 1 << KPF_MOVABLE;
+
u |= kpf_copy_bit(k, KPF_LOCKED,PG_locked);
u |= kpf_copy_bit(k, KPF_SLAB, PG_slab);
diff --git a/include/linux/fs.h b/include/linux/fs.h
index 14a97194b34b..b7ef2e41fa4a 100644
---
