Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-09 Thread Pavel Tatashin
On Thu, Jul 5, 2018 at 9:39 AM Dave Hansen wrote: > > On 07/02/2018 01:29 PM, Pavel Tatashin wrote: > > On Mon, Jul 2, 2018 at 4:00 PM Dave Hansen wrote: > >>> + unsigned long size = sizeof(struct page) * PAGES_PER_SECTION; > >>> + unsigned long pnum, map_index = 0; > >>> + void

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-09 Thread Pavel Tatashin
On Thu, Jul 5, 2018 at 9:39 AM Dave Hansen wrote: > > On 07/02/2018 01:29 PM, Pavel Tatashin wrote: > > On Mon, Jul 2, 2018 at 4:00 PM Dave Hansen wrote: > >>> + unsigned long size = sizeof(struct page) * PAGES_PER_SECTION; > >>> + unsigned long pnum, map_index = 0; > >>> + void

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-05 Thread Dave Hansen
On 07/02/2018 01:29 PM, Pavel Tatashin wrote: > On Mon, Jul 2, 2018 at 4:00 PM Dave Hansen wrote: >>> + unsigned long size = sizeof(struct page) * PAGES_PER_SECTION; >>> + unsigned long pnum, map_index = 0; >>> + void *vmemmap_buf_start; >>> + >>> + size = ALIGN(size, PMD_SIZE) *

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-05 Thread Dave Hansen
On 07/02/2018 01:29 PM, Pavel Tatashin wrote: > On Mon, Jul 2, 2018 at 4:00 PM Dave Hansen wrote: >>> + unsigned long size = sizeof(struct page) * PAGES_PER_SECTION; >>> + unsigned long pnum, map_index = 0; >>> + void *vmemmap_buf_start; >>> + >>> + size = ALIGN(size, PMD_SIZE) *

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-02 Thread Pavel Tatashin
On Mon, Jul 2, 2018 at 4:00 PM Dave Hansen wrote: > > > @@ -2651,6 +2651,14 @@ void sparse_mem_maps_populate_node(struct page > > **map_map, > > unsigned long pnum_end, > > unsigned long map_count, > >

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-02 Thread Pavel Tatashin
On Mon, Jul 2, 2018 at 4:00 PM Dave Hansen wrote: > > > @@ -2651,6 +2651,14 @@ void sparse_mem_maps_populate_node(struct page > > **map_map, > > unsigned long pnum_end, > > unsigned long map_count, > >

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-02 Thread Dave Hansen
> @@ -2651,6 +2651,14 @@ void sparse_mem_maps_populate_node(struct page > **map_map, > unsigned long pnum_end, > unsigned long map_count, > int nodeid); > +struct page *

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-02 Thread Dave Hansen
> @@ -2651,6 +2651,14 @@ void sparse_mem_maps_populate_node(struct page > **map_map, > unsigned long pnum_end, > unsigned long map_count, > int nodeid); > +struct page *

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-01 Thread Baoquan He
On 07/01/18 at 11:28pm, Pavel Tatashin wrote: > > > So, on the first failure, we even stop trying to populate other > > > sections. No more memory to do so. > > > > This is the thing I worry about. In old sparse_mem_maps_populate_node() > > you can see, when not present or failed to populate, just

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-01 Thread Baoquan He
On 07/01/18 at 11:28pm, Pavel Tatashin wrote: > > > So, on the first failure, we even stop trying to populate other > > > sections. No more memory to do so. > > > > This is the thing I worry about. In old sparse_mem_maps_populate_node() > > you can see, when not present or failed to populate, just

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-01 Thread Pavel Tatashin
> > So, on the first failure, we even stop trying to populate other > > sections. No more memory to do so. > > This is the thing I worry about. In old sparse_mem_maps_populate_node() > you can see, when not present or failed to populate, just continue. This > is the main difference between yours

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-01 Thread Pavel Tatashin
> > So, on the first failure, we even stop trying to populate other > > sections. No more memory to do so. > > This is the thing I worry about. In old sparse_mem_maps_populate_node() > you can see, when not present or failed to populate, just continue. This > is the main difference between yours

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-01 Thread Baoquan He
On 07/02/18 at 11:14am, Baoquan He wrote: > On 07/01/18 at 11:03pm, Pavel Tatashin wrote: > > > Ah, yes, I misunderstood it, sorry for that. > > > > > > Then I have only one concern, for vmemmap case, if one section doesn't > > > succeed to populate its memmap, do we need to skip all the remaining

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-01 Thread Baoquan He
On 07/02/18 at 11:14am, Baoquan He wrote: > On 07/01/18 at 11:03pm, Pavel Tatashin wrote: > > > Ah, yes, I misunderstood it, sorry for that. > > > > > > Then I have only one concern, for vmemmap case, if one section doesn't > > > succeed to populate its memmap, do we need to skip all the remaining

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-01 Thread Baoquan He
On 07/01/18 at 11:03pm, Pavel Tatashin wrote: > > Ah, yes, I misunderstood it, sorry for that. > > > > Then I have only one concern, for vmemmap case, if one section doesn't > > succeed to populate its memmap, do we need to skip all the remaining > > sections in that node? > > Yes, in

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-01 Thread Baoquan He
On 07/01/18 at 11:03pm, Pavel Tatashin wrote: > > Ah, yes, I misunderstood it, sorry for that. > > > > Then I have only one concern, for vmemmap case, if one section doesn't > > succeed to populate its memmap, do we need to skip all the remaining > > sections in that node? > > Yes, in

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-01 Thread Pavel Tatashin
> > + if (!usemap) { > > + pr_err("%s: usemap allocation failed", __func__); > > Wondering if we can provide more useful information for better debugging > if failed. E.g here tell on what nid the usemap allocation failed. > > > +

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-01 Thread Pavel Tatashin
> > + if (!usemap) { > > + pr_err("%s: usemap allocation failed", __func__); > > Wondering if we can provide more useful information for better debugging > if failed. E.g here tell on what nid the usemap allocation failed. > > > +

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-01 Thread Pavel Tatashin
> Ah, yes, I misunderstood it, sorry for that. > > Then I have only one concern, for vmemmap case, if one section doesn't > succeed to populate its memmap, do we need to skip all the remaining > sections in that node? Yes, in sparse_populate_node() we have the following: 294 for (pnum =

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-01 Thread Pavel Tatashin
> Ah, yes, I misunderstood it, sorry for that. > > Then I have only one concern, for vmemmap case, if one section doesn't > succeed to populate its memmap, do we need to skip all the remaining > sections in that node? Yes, in sparse_populate_node() we have the following: 294 for (pnum =

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-01 Thread Baoquan He
On 07/01/18 at 10:04pm, Pavel Tatashin wrote: > +/* > + * Initialize sparse on a specific node. The node spans [pnum_begin, > pnum_end) > + * And number of present sections in this node is map_count. > + */ > +void __init sparse_init_nid(int nid, unsigned long pnum_begin, > +

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-01 Thread Baoquan He
On 07/01/18 at 10:04pm, Pavel Tatashin wrote: > +/* > + * Initialize sparse on a specific node. The node spans [pnum_begin, > pnum_end) > + * And number of present sections in this node is map_count. > + */ > +void __init sparse_init_nid(int nid, unsigned long pnum_begin, > +

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-01 Thread Baoquan He
On 07/01/18 at 10:43pm, Pavel Tatashin wrote: > On Sun, Jul 1, 2018 at 10:31 PM Baoquan He wrote: > > > > On 07/01/18 at 10:18pm, Pavel Tatashin wrote: > > > > Here, I think it might be not right to jump to 'failed' directly if one > > > > section of the node failed to populate memmap. I think

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-01 Thread Baoquan He
On 07/01/18 at 10:43pm, Pavel Tatashin wrote: > On Sun, Jul 1, 2018 at 10:31 PM Baoquan He wrote: > > > > On 07/01/18 at 10:18pm, Pavel Tatashin wrote: > > > > Here, I think it might be not right to jump to 'failed' directly if one > > > > section of the node failed to populate memmap. I think

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-01 Thread Pavel Tatashin
On Sun, Jul 1, 2018 at 10:31 PM Baoquan He wrote: > > On 07/01/18 at 10:18pm, Pavel Tatashin wrote: > > > Here, I think it might be not right to jump to 'failed' directly if one > > > section of the node failed to populate memmap. I think the original code > > > is only skipping the section which

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-01 Thread Pavel Tatashin
On Sun, Jul 1, 2018 at 10:31 PM Baoquan He wrote: > > On 07/01/18 at 10:18pm, Pavel Tatashin wrote: > > > Here, I think it might be not right to jump to 'failed' directly if one > > > section of the node failed to populate memmap. I think the original code > > > is only skipping the section which

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-01 Thread Baoquan He
On 07/01/18 at 10:18pm, Pavel Tatashin wrote: > > Here, I think it might be not right to jump to 'failed' directly if one > > section of the node failed to populate memmap. I think the original code > > is only skipping the section which memmap failed to populate by marking > > it as not present

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-01 Thread Baoquan He
On 07/01/18 at 10:18pm, Pavel Tatashin wrote: > > Here, I think it might be not right to jump to 'failed' directly if one > > section of the node failed to populate memmap. I think the original code > > is only skipping the section which memmap failed to populate by marking > > it as not present

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-01 Thread Pavel Tatashin
> Here, I think it might be not right to jump to 'failed' directly if one > section of the node failed to populate memmap. I think the original code > is only skipping the section which memmap failed to populate by marking > it as not present with "ms->section_mem_map = 0". > Hi Baoquan, Thank

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-01 Thread Pavel Tatashin
> Here, I think it might be not right to jump to 'failed' directly if one > section of the node failed to populate memmap. I think the original code > is only skipping the section which memmap failed to populate by marking > it as not present with "ms->section_mem_map = 0". > Hi Baoquan, Thank

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-01 Thread Baoquan He
Hi Pavel, Thanks for your quick fix. You might have missed another comment to v2 patch 1/2 which is at the bottom. On 07/01/18 at 10:04pm, Pavel Tatashin wrote: > +/* > + * Initialize sparse on a specific node. The node spans [pnum_begin, > pnum_end) > + * And number of present sections in this

Re: [PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-01 Thread Baoquan He
Hi Pavel, Thanks for your quick fix. You might have missed another comment to v2 patch 1/2 which is at the bottom. On 07/01/18 at 10:04pm, Pavel Tatashin wrote: > +/* > + * Initialize sparse on a specific node. The node spans [pnum_begin, > pnum_end) > + * And number of present sections in this

[PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-01 Thread Pavel Tatashin
sparse_init() requires to temporary allocate two large buffers: usemap_map and map_map. Baoquan He has identified that these buffers are so large that Linux is not bootable on small memory machines, such as a kdump boot. The buffers are especially large when CONFIG_X86_5LEVEL is set, as they are

[PATCH v3 1/2] mm/sparse: add sparse_init_nid()

2018-07-01 Thread Pavel Tatashin
sparse_init() requires to temporary allocate two large buffers: usemap_map and map_map. Baoquan He has identified that these buffers are so large that Linux is not bootable on small memory machines, such as a kdump boot. The buffers are especially large when CONFIG_X86_5LEVEL is set, as they are