Re: [External] Re: [PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-18 Thread Michal Hocko
On Thu 18-02-21 11:20:51, Muchun Song wrote: > On Thu, Feb 18, 2021 at 9:00 AM Mike Kravetz wrote: > > > > On 2/17/21 12:13 AM, Michal Hocko wrote: > > > On Tue 16-02-21 11:44:34, Mike Kravetz wrote: > > > [...] > > >> If we are not going to do the allocations under the lock, then we will > > >>

Re: [External] Re: [PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-17 Thread Muchun Song
On Thu, Feb 18, 2021 at 9:00 AM Mike Kravetz wrote: > > On 2/17/21 12:13 AM, Michal Hocko wrote: > > On Tue 16-02-21 11:44:34, Mike Kravetz wrote: > > [...] > >> If we are not going to do the allocations under the lock, then we will need > >> to either preallocate or take the workqueue approach. >

Re: [External] Re: [PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-17 Thread Mike Kravetz
On 2/17/21 12:13 AM, Michal Hocko wrote: > On Tue 16-02-21 11:44:34, Mike Kravetz wrote: > [...] >> If we are not going to do the allocations under the lock, then we will need >> to either preallocate or take the workqueue approach. > > We can still drop the lock temporarily right? As we already d

Re: [External] Re: [PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-17 Thread Michal Hocko
On Tue 16-02-21 11:44:34, Mike Kravetz wrote: [...] > If we are not going to do the allocations under the lock, then we will need > to either preallocate or take the workqueue approach. We can still drop the lock temporarily right? As we already do before calling destroy_compound_gigantic_page...

Re: [External] Re: [PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-16 Thread Mike Kravetz
On 2/15/21 8:27 AM, Michal Hocko wrote: > On Mon 15-02-21 23:36:49, Muchun Song wrote: > [...] >>> There shouldn't be any real reason why the memory allocation for >>> vmemmaps, or handling vmemmap in general, has to be done from within the >>> hugetlb lock and therefore requiring a non-sleeping se

Re: [External] Re: [PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-16 Thread Muchun Song
On Tue, Feb 16, 2021 at 4:15 PM Michal Hocko wrote: > > On Tue 16-02-21 12:34:41, Muchun Song wrote: > > On Tue, Feb 16, 2021 at 3:39 AM Michal Hocko wrote: > [...] > > > > Using GFP_KERNEL will also use the current task cpuset to allocate > > > > memory. Do we have an interface to ignore current

Re: [External] Re: [PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-16 Thread David Hildenbrand
If current node has no memory and other nodes have enough memory. We can fail to allocate vmemmap pages. But actually it is suitable to allocate vmemmap pages from other nodes. Right? Falling back to a different node would be very suboptimal because then you would have vmemmap back by remote pag

Re: [External] Re: [PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-16 Thread Michal Hocko
On Tue 16-02-21 09:13:09, David Hildenbrand wrote: > On 15.02.21 20:02, Michal Hocko wrote: > > Would it be feasible to reused parts of the freed page in > > the worst case? > > As already discussed, this is only possible when the huge page does not > reside on ZONE_MOVABLE/CMA. Right. But usuall

Re: [External] Re: [PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-16 Thread Michal Hocko
On Tue 16-02-21 12:34:41, Muchun Song wrote: > On Tue, Feb 16, 2021 at 3:39 AM Michal Hocko wrote: [...] > > > Using GFP_KERNEL will also use the current task cpuset to allocate > > > memory. Do we have an interface to ignore current task cpuset?If not, > > > WQ may be the only option and it also

Re: [External] Re: [PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-16 Thread David Hildenbrand
On 15.02.21 20:02, Michal Hocko wrote: On Tue 16-02-21 01:48:29, Muchun Song wrote: On Tue, Feb 16, 2021 at 12:28 AM Michal Hocko wrote: On Mon 15-02-21 23:36:49, Muchun Song wrote: [...] There shouldn't be any real reason why the memory allocation for vmemmaps, or handling vmemmap in genera

Re: [External] Re: [PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-15 Thread Muchun Song
On Tue, Feb 16, 2021 at 3:39 AM Michal Hocko wrote: > > On Tue 16-02-21 02:19:20, Muchun Song wrote: > > On Tue, Feb 16, 2021 at 1:48 AM Muchun Song > > wrote: > > > > > > On Tue, Feb 16, 2021 at 12:28 AM Michal Hocko wrote: > > > > > > > > On Mon 15-02-21 23:36:49, Muchun Song wrote: > > > > [

Re: [External] Re: [PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-15 Thread Michal Hocko
On Tue 16-02-21 02:19:20, Muchun Song wrote: > On Tue, Feb 16, 2021 at 1:48 AM Muchun Song wrote: > > > > On Tue, Feb 16, 2021 at 12:28 AM Michal Hocko wrote: > > > > > > On Mon 15-02-21 23:36:49, Muchun Song wrote: > > > [...] > > > > > There shouldn't be any real reason why the memory allocatio

Re: [External] Re: [PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-15 Thread Michal Hocko
On Tue 16-02-21 01:48:29, Muchun Song wrote: > On Tue, Feb 16, 2021 at 12:28 AM Michal Hocko wrote: > > > > On Mon 15-02-21 23:36:49, Muchun Song wrote: > > [...] > > > > There shouldn't be any real reason why the memory allocation for > > > > vmemmaps, or handling vmemmap in general, has to be do

Re: [External] Re: [PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-15 Thread Muchun Song
On Tue, Feb 16, 2021 at 1:48 AM Muchun Song wrote: > > On Tue, Feb 16, 2021 at 12:28 AM Michal Hocko wrote: > > > > On Mon 15-02-21 23:36:49, Muchun Song wrote: > > [...] > > > > There shouldn't be any real reason why the memory allocation for > > > > vmemmaps, or handling vmemmap in general, has

Re: [External] Re: [PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-15 Thread Muchun Song
On Tue, Feb 16, 2021 at 12:28 AM Michal Hocko wrote: > > On Mon 15-02-21 23:36:49, Muchun Song wrote: > [...] > > > There shouldn't be any real reason why the memory allocation for > > > vmemmaps, or handling vmemmap in general, has to be done from within the > > > hugetlb lock and therefore requi

Re: [External] Re: [PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-15 Thread Michal Hocko
On Mon 15-02-21 23:36:49, Muchun Song wrote: [...] > > There shouldn't be any real reason why the memory allocation for > > vmemmaps, or handling vmemmap in general, has to be done from within the > > hugetlb lock and therefore requiring a non-sleeping semantic. All that > > can be deferred to a mo

Re: [External] Re: [PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-15 Thread Muchun Song
On Mon, Feb 15, 2021 at 9:19 PM Michal Hocko wrote: > > On Mon 15-02-21 20:44:57, Muchun Song wrote: > > On Mon, Feb 15, 2021 at 8:18 PM Michal Hocko wrote: > > > > > > On Mon 15-02-21 20:00:07, Muchun Song wrote: > > > > On Mon, Feb 15, 2021 at 7:51 PM Muchun Song > > > > wrote: > > > > > > >

Re: [External] Re: [PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-15 Thread Michal Hocko
On Mon 15-02-21 20:44:57, Muchun Song wrote: > On Mon, Feb 15, 2021 at 8:18 PM Michal Hocko wrote: > > > > On Mon 15-02-21 20:00:07, Muchun Song wrote: > > > On Mon, Feb 15, 2021 at 7:51 PM Muchun Song > > > wrote: > > > > > > > > On Mon, Feb 15, 2021 at 6:33 PM Michal Hocko wrote: > > > > > >

Re: [External] Re: [PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-15 Thread Muchun Song
On Mon, Feb 15, 2021 at 8:18 PM Michal Hocko wrote: > > On Mon 15-02-21 20:00:07, Muchun Song wrote: > > On Mon, Feb 15, 2021 at 7:51 PM Muchun Song > > wrote: > > > > > > On Mon, Feb 15, 2021 at 6:33 PM Michal Hocko wrote: > > > > > > > > On Mon 15-02-21 18:05:06, Muchun Song wrote: > > > > >

Re: [External] Re: [PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-15 Thread Michal Hocko
On Mon 15-02-21 19:51:26, Muchun Song wrote: > On Mon, Feb 15, 2021 at 6:33 PM Michal Hocko wrote: > > > > On Mon 15-02-21 18:05:06, Muchun Song wrote: > > > On Fri, Feb 12, 2021 at 11:32 PM Michal Hocko wrote: > > [...] > > > > > +int alloc_huge_page_vmemmap(struct hstate *h, struct page *head)

Re: [External] Re: [PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-15 Thread Michal Hocko
On Mon 15-02-21 20:00:07, Muchun Song wrote: > On Mon, Feb 15, 2021 at 7:51 PM Muchun Song wrote: > > > > On Mon, Feb 15, 2021 at 6:33 PM Michal Hocko wrote: > > > > > > On Mon 15-02-21 18:05:06, Muchun Song wrote: > > > > On Fri, Feb 12, 2021 at 11:32 PM Michal Hocko wrote: > > > [...] > > > >

Re: [External] Re: [PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-15 Thread Muchun Song
On Mon, Feb 15, 2021 at 7:51 PM Muchun Song wrote: > > On Mon, Feb 15, 2021 at 6:33 PM Michal Hocko wrote: > > > > On Mon 15-02-21 18:05:06, Muchun Song wrote: > > > On Fri, Feb 12, 2021 at 11:32 PM Michal Hocko wrote: > > [...] > > > > > +int alloc_huge_page_vmemmap(struct hstate *h, struct pag

Re: [External] Re: [PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-15 Thread Muchun Song
On Mon, Feb 15, 2021 at 6:33 PM Michal Hocko wrote: > > On Mon 15-02-21 18:05:06, Muchun Song wrote: > > On Fri, Feb 12, 2021 at 11:32 PM Michal Hocko wrote: > [...] > > > > +int alloc_huge_page_vmemmap(struct hstate *h, struct page *head) > > > > +{ > > > > + int ret; > > > > + unsigned

Re: [External] Re: [PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-15 Thread Michal Hocko
On Mon 15-02-21 18:05:06, Muchun Song wrote: > On Fri, Feb 12, 2021 at 11:32 PM Michal Hocko wrote: [...] > > > +int alloc_huge_page_vmemmap(struct hstate *h, struct page *head) > > > +{ > > > + int ret; > > > + unsigned long vmemmap_addr = (unsigned long)head; > > > + unsigned long vm

Re: [External] Re: [PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-15 Thread Muchun Song
On Fri, Feb 12, 2021 at 11:32 PM Michal Hocko wrote: > > On Mon 08-02-21 16:50:09, Muchun Song wrote: > > When we free a HugeTLB page to the buddy allocator, we should allocate the > > vmemmap pages associated with it. But we may cannot allocate vmemmap pages > > when the system is under memory pr

Re: [PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-12 Thread Michal Hocko
On Mon 08-02-21 16:50:09, Muchun Song wrote: > When we free a HugeTLB page to the buddy allocator, we should allocate the > vmemmap pages associated with it. But we may cannot allocate vmemmap pages > when the system is under memory pressure, in this case, we just refuse to > free the HugeTLB page

Re: [PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-12 Thread David Hildenbrand
On 11.02.21 19:05, Mike Kravetz wrote: On 2/8/21 12:50 AM, Muchun Song wrote: When we free a HugeTLB page to the buddy allocator, we should allocate the vmemmap pages associated with it. But we may cannot allocate vmemmap pages when the system is under memory pressure, in this case, we just refu

Re: [PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-11 Thread Mike Kravetz
On 2/8/21 12:50 AM, Muchun Song wrote: > When we free a HugeTLB page to the buddy allocator, we should allocate the > vmemmap pages associated with it. But we may cannot allocate vmemmap pages > when the system is under memory pressure, in this case, we just refuse to > free the HugeTLB page instea

[PATCH v15 4/8] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page

2021-02-08 Thread Muchun Song
When we free a HugeTLB page to the buddy allocator, we should allocate the vmemmap pages associated with it. But we may cannot allocate vmemmap pages when the system is under memory pressure, in this case, we just refuse to free the HugeTLB page instead of looping forever trying to allocate the pag