Simplify lru_add_page_tail(), there are actually only two cases possible: split_huge_page_to_list(), with list supplied and head isolated from lru by its caller; or split_huge_page(), with NULL list and head on lru - because when head is racily isolated from lru, the isolator's reference will stop the split from getting any further than its page_ref_freeze().
So decide between the two cases by "list", but add VM_WARN_ON()s to verify that they match our lru expectations. [Hugh Dickins: rewrite commit log] Signed-off-by: Alex Shi <[email protected]> Reviewed-by: Kirill A. Shutemov <[email protected]> Acked-by: Hugh Dickins <[email protected]> Cc: Kirill A. Shutemov <[email protected]> Cc: Andrew Morton <[email protected]> Cc: Johannes Weiner <[email protected]> Cc: Matthew Wilcox <[email protected]> Cc: Hugh Dickins <[email protected]> Cc: Mika Penttilä <[email protected]> Cc: [email protected] Cc: [email protected] --- mm/huge_memory.c | 20 ++++++-------------- 1 file changed, 6 insertions(+), 14 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 60726eb26840..79318d7f7d5d 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2356,24 +2356,16 @@ static void lru_add_page_tail(struct page *head, struct page *tail, VM_BUG_ON_PAGE(PageLRU(tail), head); lockdep_assert_held(&lruvec_pgdat(lruvec)->lru_lock); - if (!list) - SetPageLRU(tail); - - if (likely(PageLRU(head))) - list_add_tail(&tail->lru, &head->lru); - else if (list) { + if (list) { /* page reclaim is reclaiming a huge page */ + VM_WARN_ON(PageLRU(head)); get_page(tail); list_add_tail(&tail->lru, list); } else { - /* - * Head page has not yet been counted, as an hpage, - * so we must account for each subpage individually. - * - * Put tail on the list at the correct position - * so they all end up in order. - */ - add_page_to_lru_list_tail(tail, lruvec, page_lru(tail)); + /* head is still on lru (and we have it frozen) */ + VM_WARN_ON(!PageLRU(head)); + SetPageLRU(tail); + list_add_tail(&tail->lru, &head->lru); } } -- 1.8.3.1
