Re: [PATCH v2 0/8] fix freepage count problems in memory isolation
On Thu, Aug 07, 2014 at 08:49:00AM +0800, Zhang Yanfei wrote: > Hi Joonsoo, > > The first 3 patches in this patchset are in a bit of mess. Sorry about that. I will do better in next spin. ): Thanks. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/8] fix freepage count problems in memory isolation
Hi Joonsoo, The first 3 patches in this patchset are in a bit of mess. On 08/06/2014 03:18 PM, Joonsoo Kim wrote: > Hello, > > This patchset aims at fixing problems during memory isolation found by > testing my patchset [1]. > > These are really subtle problems so I can be wrong. If you find what I am > missing, please let me know. > > Before describing bugs itself, I first explain definition of freepage. > > 1. pages on buddy list are counted as freepage. > 2. pages on isolate migratetype buddy list are *not* counted as freepage. > 3. pages on cma buddy list are counted as CMA freepage, too. > 4. pages for guard are *not* counted as freepage. > > Now, I describe problems and related patch. > > Patch 1: If guard page are cleared and merged into isolate buddy list, > we should not add freepage count. > > Patch 4: There is race conditions that results in misplacement of free > pages on buddy list. Then, it results in incorrect freepage count and > un-availability of freepage. > > Patch 5: To count freepage correctly, we should prevent freepage from > being added to buddy list in some period of isolation. Without it, we > cannot be sure if the freepage is counted or not and miscount number > of freepage. > > Patch 7: In spite of above fixes, there is one more condition for > incorrect freepage count. pageblock isolation could be done in pageblock > unit so we can't prevent freepage from merging with page on next > pageblock. To fix it, start_isolate_page_range() and > undo_isolate_page_range() is modified to process whole range at one go. > With this change, if input parameter of start_isolate_page_range() and > undo_isolate_page_range() is properly aligned, there is no condition for > incorrect merging. > > Without patchset [1], above problem doesn't happens on my CMA allocation > test, because CMA reserved pages aren't used at all. So there is no > chance for above race. > > With patchset [1], I did simple CMA allocation test and get below result. > > - Virtual machine, 4 cpus, 1024 MB memory, 256 MB CMA reservation > - run kernel build (make -j16) on background > - 30 times CMA allocation(8MB * 30 = 240MB) attempts in 5 sec interval > - Result: more than 5000 freepage count are missed > > With patchset [1] and this patchset, I found that no freepage count are > missed so that I conclude that problems are solved. > > These problems can be possible on memory hot remove users, although > I didn't check it further. > > This patchset is based on linux-next-20140728. > Please see individual patches for more information. > > Thanks. > > [1]: Aggressively allocate the pages on cma reserved memory > https://lkml.org/lkml/2014/5/30/291 > > Joonsoo Kim (8): > mm/page_alloc: correct to clear guard attribute in DEBUG_PAGEALLOC > mm/isolation: remove unstable check for isolated page > mm/page_alloc: fix pcp high, batch management > mm/isolation: close the two race problems related to pageblock > isolation > mm/isolation: change pageblock isolation logic to fix freepage > counting bugs > mm/isolation: factor out pre/post logic on > set/unset_migratetype_isolate() > mm/isolation: fix freepage counting bug on > start/undo_isolat_page_range() > mm/isolation: remove useless race handling related to pageblock > isolation > > include/linux/page-isolation.h |2 + > mm/internal.h |5 + > mm/page_alloc.c| 223 +- > mm/page_isolation.c| 292 > +++- > 4 files changed, 368 insertions(+), 154 deletions(-) > -- Thanks. Zhang Yanfei -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v2 0/8] fix freepage count problems in memory isolation
On Wed, Aug 06, 2014 at 04:18:26PM +0900, Joonsoo Kim wrote: > Joonsoo Kim (8): > mm/page_alloc: correct to clear guard attribute in DEBUG_PAGEALLOC > mm/isolation: remove unstable check for isolated page > mm/page_alloc: fix pcp high, batch management > mm/isolation: close the two race problems related to pageblock > isolation > mm/isolation: change pageblock isolation logic to fix freepage > counting bugs > mm/isolation: factor out pre/post logic on > set/unset_migratetype_isolate() > mm/isolation: fix freepage counting bug on > start/undo_isolat_page_range() > mm/isolation: remove useless race handling related to pageblock > isolation > > include/linux/page-isolation.h |2 + > mm/internal.h |5 + > mm/page_alloc.c| 223 +- > mm/page_isolation.c| 292 > +++- > 4 files changed, 368 insertions(+), 154 deletions(-) > Sorry, Peter and Vlastimil. I missed you two on CC due to typo, so manually add on CC of cover-letter. I will do better next time. :) Thanks. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH v2 0/8] fix freepage count problems in memory isolation
Hello, This patchset aims at fixing problems during memory isolation found by testing my patchset [1]. These are really subtle problems so I can be wrong. If you find what I am missing, please let me know. Before describing bugs itself, I first explain definition of freepage. 1. pages on buddy list are counted as freepage. 2. pages on isolate migratetype buddy list are *not* counted as freepage. 3. pages on cma buddy list are counted as CMA freepage, too. 4. pages for guard are *not* counted as freepage. Now, I describe problems and related patch. Patch 1: If guard page are cleared and merged into isolate buddy list, we should not add freepage count. Patch 4: There is race conditions that results in misplacement of free pages on buddy list. Then, it results in incorrect freepage count and un-availability of freepage. Patch 5: To count freepage correctly, we should prevent freepage from being added to buddy list in some period of isolation. Without it, we cannot be sure if the freepage is counted or not and miscount number of freepage. Patch 7: In spite of above fixes, there is one more condition for incorrect freepage count. pageblock isolation could be done in pageblock unit so we can't prevent freepage from merging with page on next pageblock. To fix it, start_isolate_page_range() and undo_isolate_page_range() is modified to process whole range at one go. With this change, if input parameter of start_isolate_page_range() and undo_isolate_page_range() is properly aligned, there is no condition for incorrect merging. Without patchset [1], above problem doesn't happens on my CMA allocation test, because CMA reserved pages aren't used at all. So there is no chance for above race. With patchset [1], I did simple CMA allocation test and get below result. - Virtual machine, 4 cpus, 1024 MB memory, 256 MB CMA reservation - run kernel build (make -j16) on background - 30 times CMA allocation(8MB * 30 = 240MB) attempts in 5 sec interval - Result: more than 5000 freepage count are missed With patchset [1] and this patchset, I found that no freepage count are missed so that I conclude that problems are solved. These problems can be possible on memory hot remove users, although I didn't check it further. This patchset is based on linux-next-20140728. Please see individual patches for more information. Thanks. [1]: Aggressively allocate the pages on cma reserved memory https://lkml.org/lkml/2014/5/30/291 Joonsoo Kim (8): mm/page_alloc: correct to clear guard attribute in DEBUG_PAGEALLOC mm/isolation: remove unstable check for isolated page mm/page_alloc: fix pcp high, batch management mm/isolation: close the two race problems related to pageblock isolation mm/isolation: change pageblock isolation logic to fix freepage counting bugs mm/isolation: factor out pre/post logic on set/unset_migratetype_isolate() mm/isolation: fix freepage counting bug on start/undo_isolat_page_range() mm/isolation: remove useless race handling related to pageblock isolation include/linux/page-isolation.h |2 + mm/internal.h |5 + mm/page_alloc.c| 223 +- mm/page_isolation.c| 292 +++- 4 files changed, 368 insertions(+), 154 deletions(-) -- 1.7.9.5 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
