Re: [PATCH v2 4/5] mm: memory_hotplug: Add memory hotremove probe device

2017-12-04 Thread Reza Arbab 

On Fri, Nov 24, 2017 at 02:29:48PM +, Andrea Reale wrote:

But, at least in my understanding, the implementation is not as
straightfoward as it looks. If I declare a memory node in the fdt, then,
at boot, the kernel will expect that memory to actually be there to be
used: this is not true if I want to plug my dimms only later at runtime.
So I think that declaring the hotpluggable memory in an fdt memory
node might not feasible without changes.


On the power arch, we do this today using "linux,usable-memory".

memory@100 {
 device_type = "memory";
 reg = <0x100 0x0 0x0 0x8000>;
 linux,usable-memory = <0x100 0x0 0x0 0x4000>;
 :
}

The reg range defines the node, but at at boot, memblocks are only 
created for the linux,usable-memory range. The rest can be hotplugged 
later. YMMV, because this depends on your arch's implementation of 
memory_add_physaddr_to_nid().


One idea could be to add a new property to memory nodes, to specify 
what memory is potentially hotplugguable.


Somewhat related, there is already a "hotpluggable" property.

memory@1004000 {
 device_type = "memory";
 reg = <0x100 0x4000 0x0 0x4000>;
 hotpluggable;
 :
}

This is subtly different from the earlier example. This memory IS 
present at boot. The hotpluggable property ensures that it resides in 
ZONE_MOVABLE so it can potentially be removed.


--
Reza Arbab

Re: [PATCH v2 4/5] mm: memory_hotplug: Add memory hotremove probe device

2017-12-04 Thread Reza Arbab 

On Fri, Nov 24, 2017 at 02:29:48PM +, Andrea Reale wrote:

But, at least in my understanding, the implementation is not as
straightfoward as it looks. If I declare a memory node in the fdt, then,
at boot, the kernel will expect that memory to actually be there to be
used: this is not true if I want to plug my dimms only later at runtime.
So I think that declaring the hotpluggable memory in an fdt memory
node might not feasible without changes.


On the power arch, we do this today using "linux,usable-memory".

memory@100 {
 device_type = "memory";
 reg = <0x100 0x0 0x0 0x8000>;
 linux,usable-memory = <0x100 0x0 0x0 0x4000>;
 :
}

The reg range defines the node, but at at boot, memblocks are only 
created for the linux,usable-memory range. The rest can be hotplugged 
later. YMMV, because this depends on your arch's implementation of 
memory_add_physaddr_to_nid().


One idea could be to add a new property to memory nodes, to specify 
what memory is potentially hotplugguable.


Somewhat related, there is already a "hotpluggable" property.

memory@1004000 {
 device_type = "memory";
 reg = <0x100 0x4000 0x0 0x4000>;
 hotpluggable;
 :
}

This is subtly different from the earlier example. This memory IS 
present at boot. The hotpluggable property ensures that it resides in 
ZONE_MOVABLE so it can potentially be removed.


--
Reza Arbab

Re: [PATCH v2 4/5] mm: memory_hotplug: Add memory hotremove probe device

2017-12-04 Thread Andrea Reale 
On Mon  4 Dec 2017, 13:33, Michal Hocko wrote:
> On Mon 04-12-17 11:51:29, Andrea Reale wrote:
> > On Thu 30 Nov 2017, 15:49, Michal Hocko wrote:
> > > On Thu 23-11-17 11:14:52, Andrea Reale wrote:
> > > > Adding a "remove" sysfs handle that can be used to trigger
> > > > memory hotremove manually, exactly simmetrically with
> > > > what happens with the "probe" device for hot-add.
> > > > 
> > > > This is usueful for architecture that do not rely on
> > > > ACPI for memory hot-remove.
> > > 
> > > As already said elsewhere, this really has to check the online status of
> > > the range and fail some is still online.
> > > 
> > 
> > This is actually still done in remove_memory() (patch 2/5) with
> > walk_memory_range. We just return an error rather than BUGing().
> > 
> > Or are you referring to something else?
> 
> But you are not returning that error to the caller, are you?
> 
> [...]

Oh, I see your point. Yes, indeed we should have returned it. Thanks for
catching the issue.

> > > > +   nid = memory_add_physaddr_to_nid(phys_addr);
> > > > +   ret = lock_device_hotplug_sysfs();
> > > > +   if (ret)
> > > > +   return ret;
> > > > +
> > > > +   remove_memory(nid, phys_addr,
> > > > +MIN_MEMORY_BLOCK_SIZE * sections_per_block);
> > > > +   unlock_device_hotplug();
> > > > +   return count;

Thanks,
Andrea
> 
> -- 
> Michal Hocko
> SUSE Labs
>

Re: [PATCH v2 4/5] mm: memory_hotplug: Add memory hotremove probe device

2017-12-04 Thread Andrea Reale 
On Mon  4 Dec 2017, 13:33, Michal Hocko wrote:
> On Mon 04-12-17 11:51:29, Andrea Reale wrote:
> > On Thu 30 Nov 2017, 15:49, Michal Hocko wrote:
> > > On Thu 23-11-17 11:14:52, Andrea Reale wrote:
> > > > Adding a "remove" sysfs handle that can be used to trigger
> > > > memory hotremove manually, exactly simmetrically with
> > > > what happens with the "probe" device for hot-add.
> > > > 
> > > > This is usueful for architecture that do not rely on
> > > > ACPI for memory hot-remove.
> > > 
> > > As already said elsewhere, this really has to check the online status of
> > > the range and fail some is still online.
> > > 
> > 
> > This is actually still done in remove_memory() (patch 2/5) with
> > walk_memory_range. We just return an error rather than BUGing().
> > 
> > Or are you referring to something else?
> 
> But you are not returning that error to the caller, are you?
> 
> [...]

Oh, I see your point. Yes, indeed we should have returned it. Thanks for
catching the issue.

> > > > +   nid = memory_add_physaddr_to_nid(phys_addr);
> > > > +   ret = lock_device_hotplug_sysfs();
> > > > +   if (ret)
> > > > +   return ret;
> > > > +
> > > > +   remove_memory(nid, phys_addr,
> > > > +MIN_MEMORY_BLOCK_SIZE * sections_per_block);
> > > > +   unlock_device_hotplug();
> > > > +   return count;

Thanks,
Andrea
> 
> -- 
> Michal Hocko
> SUSE Labs
>

Re: [PATCH v2 4/5] mm: memory_hotplug: Add memory hotremove probe device

2017-12-04 Thread Michal Hocko 
On Mon 04-12-17 11:51:29, Andrea Reale wrote:
> On Thu 30 Nov 2017, 15:49, Michal Hocko wrote:
> > On Thu 23-11-17 11:14:52, Andrea Reale wrote:
> > > Adding a "remove" sysfs handle that can be used to trigger
> > > memory hotremove manually, exactly simmetrically with
> > > what happens with the "probe" device for hot-add.
> > > 
> > > This is usueful for architecture that do not rely on
> > > ACPI for memory hot-remove.
> > 
> > As already said elsewhere, this really has to check the online status of
> > the range and fail some is still online.
> > 
> 
> This is actually still done in remove_memory() (patch 2/5) with
> walk_memory_range. We just return an error rather than BUGing().
> 
> Or are you referring to something else?

But you are not returning that error to the caller, are you?

[...]
> > > + nid = memory_add_physaddr_to_nid(phys_addr);
> > > + ret = lock_device_hotplug_sysfs();
> > > + if (ret)
> > > + return ret;
> > > +
> > > + remove_memory(nid, phys_addr,
> > > +  MIN_MEMORY_BLOCK_SIZE * sections_per_block);
> > > + unlock_device_hotplug();
> > > + return count;

-- 
Michal Hocko
SUSE Labs

Re: [PATCH v2 4/5] mm: memory_hotplug: Add memory hotremove probe device

2017-12-04 Thread Michal Hocko 
On Mon 04-12-17 11:51:29, Andrea Reale wrote:
> On Thu 30 Nov 2017, 15:49, Michal Hocko wrote:
> > On Thu 23-11-17 11:14:52, Andrea Reale wrote:
> > > Adding a "remove" sysfs handle that can be used to trigger
> > > memory hotremove manually, exactly simmetrically with
> > > what happens with the "probe" device for hot-add.
> > > 
> > > This is usueful for architecture that do not rely on
> > > ACPI for memory hot-remove.
> > 
> > As already said elsewhere, this really has to check the online status of
> > the range and fail some is still online.
> > 
> 
> This is actually still done in remove_memory() (patch 2/5) with
> walk_memory_range. We just return an error rather than BUGing().
> 
> Or are you referring to something else?

But you are not returning that error to the caller, are you?

[...]
> > > + nid = memory_add_physaddr_to_nid(phys_addr);
> > > + ret = lock_device_hotplug_sysfs();
> > > + if (ret)
> > > + return ret;
> > > +
> > > + remove_memory(nid, phys_addr,
> > > +  MIN_MEMORY_BLOCK_SIZE * sections_per_block);
> > > + unlock_device_hotplug();
> > > + return count;

-- 
Michal Hocko
SUSE Labs

Re: [PATCH v2 4/5] mm: memory_hotplug: Add memory hotremove probe device

2017-12-04 Thread Andrea Reale 
On Thu 30 Nov 2017, 15:49, Michal Hocko wrote:
> On Thu 23-11-17 11:14:52, Andrea Reale wrote:
> > Adding a "remove" sysfs handle that can be used to trigger
> > memory hotremove manually, exactly simmetrically with
> > what happens with the "probe" device for hot-add.
> > 
> > This is usueful for architecture that do not rely on
> > ACPI for memory hot-remove.
> 
> As already said elsewhere, this really has to check the online status of
> the range and fail some is still online.
> 

This is actually still done in remove_memory() (patch 2/5) with
walk_memory_range. We just return an error rather than BUGing().

Or are you referring to something else?


> > Signed-off-by: Andrea Reale 
> > Signed-off-by: Maciej Bielski 
> > ---
> >  drivers/base/memory.c | 34 +-
> >  1 file changed, 33 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/base/memory.c b/drivers/base/memory.c
> > index 1d60b58..8ccb67c 100644
> > --- a/drivers/base/memory.c
> > +++ b/drivers/base/memory.c
> > @@ -530,7 +530,36 @@ memory_probe_store(struct device *dev, struct 
> > device_attribute *attr,
> >  }
> >  
> >  static DEVICE_ATTR(probe, S_IWUSR, NULL, memory_probe_store);
> > -#endif
> > +
> > +#ifdef CONFIG_MEMORY_HOTREMOVE
> > +static ssize_t
> > +memory_remove_store(struct device *dev,
> > +   struct device_attribute *attr, const char *buf, size_t count)
> > +{
> > +   u64 phys_addr;
> > +   int nid, ret;
> > +   unsigned long pages_per_block = PAGES_PER_SECTION * sections_per_block;
> > +
> > +   ret = kstrtoull(buf, 0, _addr);
> > +   if (ret)
> > +   return ret;
> > +
> > +   if (phys_addr & ((pages_per_block << PAGE_SHIFT) - 1))
> > +   return -EINVAL;
> > +
> > +   nid = memory_add_physaddr_to_nid(phys_addr);
> > +   ret = lock_device_hotplug_sysfs();
> > +   if (ret)
> > +   return ret;
> > +
> > +   remove_memory(nid, phys_addr,
> > +MIN_MEMORY_BLOCK_SIZE * sections_per_block);
> > +   unlock_device_hotplug();
> > +   return count;
> > +}
> > +static DEVICE_ATTR(remove, S_IWUSR, NULL, memory_remove_store);
> > +#endif /* CONFIG_MEMORY_HOTREMOVE */
> > +#endif /* CONFIG_ARCH_MEMORY_PROBE */
> >  
> >  #ifdef CONFIG_MEMORY_FAILURE
> >  /*
> > @@ -790,6 +819,9 @@ bool is_memblock_offlined(struct memory_block *mem)
> >  static struct attribute *memory_root_attrs[] = {
> >  #ifdef CONFIG_ARCH_MEMORY_PROBE
> > _attr_probe.attr,
> > +#ifdef CONFIG_MEMORY_HOTREMOVE
> > +   _attr_remove.attr,
> > +#endif
> >  #endif
> >  
> >  #ifdef CONFIG_MEMORY_FAILURE
> > -- 
> > 2.7.4

Thanks,
Andrea

> 
> -- 
> Michal Hocko
> SUSE Labs
>

Re: [PATCH v2 4/5] mm: memory_hotplug: Add memory hotremove probe device

2017-12-04 Thread Andrea Reale 
On Thu 30 Nov 2017, 15:49, Michal Hocko wrote:
> On Thu 23-11-17 11:14:52, Andrea Reale wrote:
> > Adding a "remove" sysfs handle that can be used to trigger
> > memory hotremove manually, exactly simmetrically with
> > what happens with the "probe" device for hot-add.
> > 
> > This is usueful for architecture that do not rely on
> > ACPI for memory hot-remove.
> 
> As already said elsewhere, this really has to check the online status of
> the range and fail some is still online.
> 

This is actually still done in remove_memory() (patch 2/5) with
walk_memory_range. We just return an error rather than BUGing().

Or are you referring to something else?


> > Signed-off-by: Andrea Reale 
> > Signed-off-by: Maciej Bielski 
> > ---
> >  drivers/base/memory.c | 34 +-
> >  1 file changed, 33 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/base/memory.c b/drivers/base/memory.c
> > index 1d60b58..8ccb67c 100644
> > --- a/drivers/base/memory.c
> > +++ b/drivers/base/memory.c
> > @@ -530,7 +530,36 @@ memory_probe_store(struct device *dev, struct 
> > device_attribute *attr,
> >  }
> >  
> >  static DEVICE_ATTR(probe, S_IWUSR, NULL, memory_probe_store);
> > -#endif
> > +
> > +#ifdef CONFIG_MEMORY_HOTREMOVE
> > +static ssize_t
> > +memory_remove_store(struct device *dev,
> > +   struct device_attribute *attr, const char *buf, size_t count)
> > +{
> > +   u64 phys_addr;
> > +   int nid, ret;
> > +   unsigned long pages_per_block = PAGES_PER_SECTION * sections_per_block;
> > +
> > +   ret = kstrtoull(buf, 0, _addr);
> > +   if (ret)
> > +   return ret;
> > +
> > +   if (phys_addr & ((pages_per_block << PAGE_SHIFT) - 1))
> > +   return -EINVAL;
> > +
> > +   nid = memory_add_physaddr_to_nid(phys_addr);
> > +   ret = lock_device_hotplug_sysfs();
> > +   if (ret)
> > +   return ret;
> > +
> > +   remove_memory(nid, phys_addr,
> > +MIN_MEMORY_BLOCK_SIZE * sections_per_block);
> > +   unlock_device_hotplug();
> > +   return count;
> > +}
> > +static DEVICE_ATTR(remove, S_IWUSR, NULL, memory_remove_store);
> > +#endif /* CONFIG_MEMORY_HOTREMOVE */
> > +#endif /* CONFIG_ARCH_MEMORY_PROBE */
> >  
> >  #ifdef CONFIG_MEMORY_FAILURE
> >  /*
> > @@ -790,6 +819,9 @@ bool is_memblock_offlined(struct memory_block *mem)
> >  static struct attribute *memory_root_attrs[] = {
> >  #ifdef CONFIG_ARCH_MEMORY_PROBE
> > _attr_probe.attr,
> > +#ifdef CONFIG_MEMORY_HOTREMOVE
> > +   _attr_remove.attr,
> > +#endif
> >  #endif
> >  
> >  #ifdef CONFIG_MEMORY_FAILURE
> > -- 
> > 2.7.4

Thanks,
Andrea

> 
> -- 
> Michal Hocko
> SUSE Labs
>

Re: [PATCH v2 4/5] mm: memory_hotplug: Add memory hotremove probe device

2017-11-30 Thread Michal Hocko 
On Thu 23-11-17 11:14:52, Andrea Reale wrote:
> Adding a "remove" sysfs handle that can be used to trigger
> memory hotremove manually, exactly simmetrically with
> what happens with the "probe" device for hot-add.
> 
> This is usueful for architecture that do not rely on
> ACPI for memory hot-remove.

As already said elsewhere, this really has to check the online status of
the range and fail some is still online.

> Signed-off-by: Andrea Reale 
> Signed-off-by: Maciej Bielski 
> ---
>  drivers/base/memory.c | 34 +-
>  1 file changed, 33 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/base/memory.c b/drivers/base/memory.c
> index 1d60b58..8ccb67c 100644
> --- a/drivers/base/memory.c
> +++ b/drivers/base/memory.c
> @@ -530,7 +530,36 @@ memory_probe_store(struct device *dev, struct 
> device_attribute *attr,
>  }
>  
>  static DEVICE_ATTR(probe, S_IWUSR, NULL, memory_probe_store);
> -#endif
> +
> +#ifdef CONFIG_MEMORY_HOTREMOVE
> +static ssize_t
> +memory_remove_store(struct device *dev,
> + struct device_attribute *attr, const char *buf, size_t count)
> +{
> + u64 phys_addr;
> + int nid, ret;
> + unsigned long pages_per_block = PAGES_PER_SECTION * sections_per_block;
> +
> + ret = kstrtoull(buf, 0, _addr);
> + if (ret)
> + return ret;
> +
> + if (phys_addr & ((pages_per_block << PAGE_SHIFT) - 1))
> + return -EINVAL;
> +
> + nid = memory_add_physaddr_to_nid(phys_addr);
> + ret = lock_device_hotplug_sysfs();
> + if (ret)
> + return ret;
> +
> + remove_memory(nid, phys_addr,
> +  MIN_MEMORY_BLOCK_SIZE * sections_per_block);
> + unlock_device_hotplug();
> + return count;
> +}
> +static DEVICE_ATTR(remove, S_IWUSR, NULL, memory_remove_store);
> +#endif /* CONFIG_MEMORY_HOTREMOVE */
> +#endif /* CONFIG_ARCH_MEMORY_PROBE */
>  
>  #ifdef CONFIG_MEMORY_FAILURE
>  /*
> @@ -790,6 +819,9 @@ bool is_memblock_offlined(struct memory_block *mem)
>  static struct attribute *memory_root_attrs[] = {
>  #ifdef CONFIG_ARCH_MEMORY_PROBE
>   _attr_probe.attr,
> +#ifdef CONFIG_MEMORY_HOTREMOVE
> + _attr_remove.attr,
> +#endif
>  #endif
>  
>  #ifdef CONFIG_MEMORY_FAILURE
> -- 
> 2.7.4

-- 
Michal Hocko
SUSE Labs

Re: [PATCH v2 4/5] mm: memory_hotplug: Add memory hotremove probe device

2017-11-30 Thread Michal Hocko 
On Thu 23-11-17 11:14:52, Andrea Reale wrote:
> Adding a "remove" sysfs handle that can be used to trigger
> memory hotremove manually, exactly simmetrically with
> what happens with the "probe" device for hot-add.
> 
> This is usueful for architecture that do not rely on
> ACPI for memory hot-remove.

As already said elsewhere, this really has to check the online status of
the range and fail some is still online.

> Signed-off-by: Andrea Reale 
> Signed-off-by: Maciej Bielski 
> ---
>  drivers/base/memory.c | 34 +-
>  1 file changed, 33 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/base/memory.c b/drivers/base/memory.c
> index 1d60b58..8ccb67c 100644
> --- a/drivers/base/memory.c
> +++ b/drivers/base/memory.c
> @@ -530,7 +530,36 @@ memory_probe_store(struct device *dev, struct 
> device_attribute *attr,
>  }
>  
>  static DEVICE_ATTR(probe, S_IWUSR, NULL, memory_probe_store);
> -#endif
> +
> +#ifdef CONFIG_MEMORY_HOTREMOVE
> +static ssize_t
> +memory_remove_store(struct device *dev,
> + struct device_attribute *attr, const char *buf, size_t count)
> +{
> + u64 phys_addr;
> + int nid, ret;
> + unsigned long pages_per_block = PAGES_PER_SECTION * sections_per_block;
> +
> + ret = kstrtoull(buf, 0, _addr);
> + if (ret)
> + return ret;
> +
> + if (phys_addr & ((pages_per_block << PAGE_SHIFT) - 1))
> + return -EINVAL;
> +
> + nid = memory_add_physaddr_to_nid(phys_addr);
> + ret = lock_device_hotplug_sysfs();
> + if (ret)
> + return ret;
> +
> + remove_memory(nid, phys_addr,
> +  MIN_MEMORY_BLOCK_SIZE * sections_per_block);
> + unlock_device_hotplug();
> + return count;
> +}
> +static DEVICE_ATTR(remove, S_IWUSR, NULL, memory_remove_store);
> +#endif /* CONFIG_MEMORY_HOTREMOVE */
> +#endif /* CONFIG_ARCH_MEMORY_PROBE */
>  
>  #ifdef CONFIG_MEMORY_FAILURE
>  /*
> @@ -790,6 +819,9 @@ bool is_memblock_offlined(struct memory_block *mem)
>  static struct attribute *memory_root_attrs[] = {
>  #ifdef CONFIG_ARCH_MEMORY_PROBE
>   _attr_probe.attr,
> +#ifdef CONFIG_MEMORY_HOTREMOVE
> + _attr_remove.attr,
> +#endif
>  #endif
>  
>  #ifdef CONFIG_MEMORY_FAILURE
> -- 
> 2.7.4

-- 
Michal Hocko
SUSE Labs

Re: [PATCH v2 4/5] mm: memory_hotplug: Add memory hotremove probe device

2017-11-27 Thread Andrea Reale 
Hi Robin,

On Mon 27 Nov 2017, 15:33, Robin Murphy wrote:
> On 23/11/17 11:14, Andrea Reale wrote:
> >Adding a "remove" sysfs handle that can be used to trigger
> >memory hotremove manually, exactly simmetrically with
> >what happens with the "probe" device for hot-add.
> >
> >This is usueful for architecture that do not rely on
> >ACPI for memory hot-remove.
> 
> Is there a real-world use-case for this, or is it mostly just a handy
> development feature?
> 
as I was saying in a response to your previous message, in our use
case remove events are triggered by software. Besides our use case,
yes, it is mostly just a handy develeopment feature AFAICT.

> >Signed-off-by: Andrea Reale 
> >Signed-off-by: Maciej Bielski 
> >---
> >  drivers/base/memory.c | 34 +-
> >  1 file changed, 33 insertions(+), 1 deletion(-)
> >
> >diff --git a/drivers/base/memory.c b/drivers/base/memory.c
> >index 1d60b58..8ccb67c 100644
> >--- a/drivers/base/memory.c
> >+++ b/drivers/base/memory.c
> >@@ -530,7 +530,36 @@ memory_probe_store(struct device *dev, struct 
> >device_attribute *attr,
> >  }
> >  static DEVICE_ATTR(probe, S_IWUSR, NULL, memory_probe_store);
> >-#endif
> >+
> >+#ifdef CONFIG_MEMORY_HOTREMOVE
> >+static ssize_t
> >+memory_remove_store(struct device *dev,
> >+struct device_attribute *attr, const char *buf, size_t count)
> >+{
> >+u64 phys_addr;
> >+int nid, ret;
> >+unsigned long pages_per_block = PAGES_PER_SECTION * sections_per_block;
> >+
> >+ret = kstrtoull(buf, 0, _addr);
> >+if (ret)
> >+return ret;
> >+
> >+if (phys_addr & ((pages_per_block << PAGE_SHIFT) - 1))
> >+return -EINVAL;
> >+
> >+nid = memory_add_physaddr_to_nid(phys_addr);
> 
> This call looks a bit odd, since you're not doing a memory add. In fact, any
> memory being removed should already be fully known-about, so AFAICS it
> should be simple to get everything you need to know (including potentially
> the online status as mentioned earlier), through 'normal' methods, e.g.
> page_to_nid() or similar.

Makes sense. Suggestion noted, thanks.

> Robin.
> 
> >+ret = lock_device_hotplug_sysfs();
> >+if (ret)
> >+return ret;
> >+
> >+remove_memory(nid, phys_addr,
> >+ MIN_MEMORY_BLOCK_SIZE * sections_per_block);
> >+unlock_device_hotplug();
> >+return count;
> >+}
> >+static DEVICE_ATTR(remove, S_IWUSR, NULL, memory_remove_store);
> >+#endif /* CONFIG_MEMORY_HOTREMOVE */
> >+#endif /* CONFIG_ARCH_MEMORY_PROBE */
> >  #ifdef CONFIG_MEMORY_FAILURE
> >  /*
> >@@ -790,6 +819,9 @@ bool is_memblock_offlined(struct memory_block *mem)
> >  static struct attribute *memory_root_attrs[] = {
> >  #ifdef CONFIG_ARCH_MEMORY_PROBE
> > _attr_probe.attr,
> >+#ifdef CONFIG_MEMORY_HOTREMOVE
> >+_attr_remove.attr,
> >+#endif
> >  #endif
> >  #ifdef CONFIG_MEMORY_FAILURE
> >
>

Re: [PATCH v2 4/5] mm: memory_hotplug: Add memory hotremove probe device

2017-11-27 Thread Andrea Reale 
Hi Robin,

On Mon 27 Nov 2017, 15:33, Robin Murphy wrote:
> On 23/11/17 11:14, Andrea Reale wrote:
> >Adding a "remove" sysfs handle that can be used to trigger
> >memory hotremove manually, exactly simmetrically with
> >what happens with the "probe" device for hot-add.
> >
> >This is usueful for architecture that do not rely on
> >ACPI for memory hot-remove.
> 
> Is there a real-world use-case for this, or is it mostly just a handy
> development feature?
> 
as I was saying in a response to your previous message, in our use
case remove events are triggered by software. Besides our use case,
yes, it is mostly just a handy develeopment feature AFAICT.

> >Signed-off-by: Andrea Reale 
> >Signed-off-by: Maciej Bielski 
> >---
> >  drivers/base/memory.c | 34 +-
> >  1 file changed, 33 insertions(+), 1 deletion(-)
> >
> >diff --git a/drivers/base/memory.c b/drivers/base/memory.c
> >index 1d60b58..8ccb67c 100644
> >--- a/drivers/base/memory.c
> >+++ b/drivers/base/memory.c
> >@@ -530,7 +530,36 @@ memory_probe_store(struct device *dev, struct 
> >device_attribute *attr,
> >  }
> >  static DEVICE_ATTR(probe, S_IWUSR, NULL, memory_probe_store);
> >-#endif
> >+
> >+#ifdef CONFIG_MEMORY_HOTREMOVE
> >+static ssize_t
> >+memory_remove_store(struct device *dev,
> >+struct device_attribute *attr, const char *buf, size_t count)
> >+{
> >+u64 phys_addr;
> >+int nid, ret;
> >+unsigned long pages_per_block = PAGES_PER_SECTION * sections_per_block;
> >+
> >+ret = kstrtoull(buf, 0, _addr);
> >+if (ret)
> >+return ret;
> >+
> >+if (phys_addr & ((pages_per_block << PAGE_SHIFT) - 1))
> >+return -EINVAL;
> >+
> >+nid = memory_add_physaddr_to_nid(phys_addr);
> 
> This call looks a bit odd, since you're not doing a memory add. In fact, any
> memory being removed should already be fully known-about, so AFAICS it
> should be simple to get everything you need to know (including potentially
> the online status as mentioned earlier), through 'normal' methods, e.g.
> page_to_nid() or similar.

Makes sense. Suggestion noted, thanks.

> Robin.
> 
> >+ret = lock_device_hotplug_sysfs();
> >+if (ret)
> >+return ret;
> >+
> >+remove_memory(nid, phys_addr,
> >+ MIN_MEMORY_BLOCK_SIZE * sections_per_block);
> >+unlock_device_hotplug();
> >+return count;
> >+}
> >+static DEVICE_ATTR(remove, S_IWUSR, NULL, memory_remove_store);
> >+#endif /* CONFIG_MEMORY_HOTREMOVE */
> >+#endif /* CONFIG_ARCH_MEMORY_PROBE */
> >  #ifdef CONFIG_MEMORY_FAILURE
> >  /*
> >@@ -790,6 +819,9 @@ bool is_memblock_offlined(struct memory_block *mem)
> >  static struct attribute *memory_root_attrs[] = {
> >  #ifdef CONFIG_ARCH_MEMORY_PROBE
> > _attr_probe.attr,
> >+#ifdef CONFIG_MEMORY_HOTREMOVE
> >+_attr_remove.attr,
> >+#endif
> >  #endif
> >  #ifdef CONFIG_MEMORY_FAILURE
> >
>

Re: [PATCH v2 4/5] mm: memory_hotplug: Add memory hotremove probe device

2017-11-27 Thread Robin Murphy 

On 23/11/17 11:14, Andrea Reale wrote:

Adding a "remove" sysfs handle that can be used to trigger
memory hotremove manually, exactly simmetrically with
what happens with the "probe" device for hot-add.

This is usueful for architecture that do not rely on
ACPI for memory hot-remove.


Is there a real-world use-case for this, or is it mostly just a handy 
development feature?



Signed-off-by: Andrea Reale 
Signed-off-by: Maciej Bielski 
---
  drivers/base/memory.c | 34 +-
  1 file changed, 33 insertions(+), 1 deletion(-)

diff --git a/drivers/base/memory.c b/drivers/base/memory.c
index 1d60b58..8ccb67c 100644
--- a/drivers/base/memory.c
+++ b/drivers/base/memory.c
@@ -530,7 +530,36 @@ memory_probe_store(struct device *dev, struct 
device_attribute *attr,
  }
  
  static DEVICE_ATTR(probe, S_IWUSR, NULL, memory_probe_store);

-#endif
+
+#ifdef CONFIG_MEMORY_HOTREMOVE
+static ssize_t
+memory_remove_store(struct device *dev,
+   struct device_attribute *attr, const char *buf, size_t count)
+{
+   u64 phys_addr;
+   int nid, ret;
+   unsigned long pages_per_block = PAGES_PER_SECTION * sections_per_block;
+
+   ret = kstrtoull(buf, 0, _addr);
+   if (ret)
+   return ret;
+
+   if (phys_addr & ((pages_per_block << PAGE_SHIFT) - 1))
+   return -EINVAL;
+
+   nid = memory_add_physaddr_to_nid(phys_addr);


This call looks a bit odd, since you're not doing a memory add. In fact, 
any memory being removed should already be fully known-about, so AFAICS 
it should be simple to get everything you need to know (including 
potentially the online status as mentioned earlier), through 'normal' 
methods, e.g. page_to_nid() or similar.


Robin.


+   ret = lock_device_hotplug_sysfs();
+   if (ret)
+   return ret;
+
+   remove_memory(nid, phys_addr,
+MIN_MEMORY_BLOCK_SIZE * sections_per_block);
+   unlock_device_hotplug();
+   return count;
+}
+static DEVICE_ATTR(remove, S_IWUSR, NULL, memory_remove_store);
+#endif /* CONFIG_MEMORY_HOTREMOVE */
+#endif /* CONFIG_ARCH_MEMORY_PROBE */
  
  #ifdef CONFIG_MEMORY_FAILURE

  /*
@@ -790,6 +819,9 @@ bool is_memblock_offlined(struct memory_block *mem)
  static struct attribute *memory_root_attrs[] = {
  #ifdef CONFIG_ARCH_MEMORY_PROBE
_attr_probe.attr,
+#ifdef CONFIG_MEMORY_HOTREMOVE
+   _attr_remove.attr,
+#endif
  #endif
  
  #ifdef CONFIG_MEMORY_FAILURE

Re: [PATCH v2 4/5] mm: memory_hotplug: Add memory hotremove probe device

2017-11-27 Thread Robin Murphy 

On 23/11/17 11:14, Andrea Reale wrote:

Adding a "remove" sysfs handle that can be used to trigger
memory hotremove manually, exactly simmetrically with
what happens with the "probe" device for hot-add.

This is usueful for architecture that do not rely on
ACPI for memory hot-remove.


Is there a real-world use-case for this, or is it mostly just a handy 
development feature?



Signed-off-by: Andrea Reale 
Signed-off-by: Maciej Bielski 
---
  drivers/base/memory.c | 34 +-
  1 file changed, 33 insertions(+), 1 deletion(-)

diff --git a/drivers/base/memory.c b/drivers/base/memory.c
index 1d60b58..8ccb67c 100644
--- a/drivers/base/memory.c
+++ b/drivers/base/memory.c
@@ -530,7 +530,36 @@ memory_probe_store(struct device *dev, struct 
device_attribute *attr,
  }
  
  static DEVICE_ATTR(probe, S_IWUSR, NULL, memory_probe_store);

-#endif
+
+#ifdef CONFIG_MEMORY_HOTREMOVE
+static ssize_t
+memory_remove_store(struct device *dev,
+   struct device_attribute *attr, const char *buf, size_t count)
+{
+   u64 phys_addr;
+   int nid, ret;
+   unsigned long pages_per_block = PAGES_PER_SECTION * sections_per_block;
+
+   ret = kstrtoull(buf, 0, _addr);
+   if (ret)
+   return ret;
+
+   if (phys_addr & ((pages_per_block << PAGE_SHIFT) - 1))
+   return -EINVAL;
+
+   nid = memory_add_physaddr_to_nid(phys_addr);


This call looks a bit odd, since you're not doing a memory add. In fact, 
any memory being removed should already be fully known-about, so AFAICS 
it should be simple to get everything you need to know (including 
potentially the online status as mentioned earlier), through 'normal' 
methods, e.g. page_to_nid() or similar.


Robin.


+   ret = lock_device_hotplug_sysfs();
+   if (ret)
+   return ret;
+
+   remove_memory(nid, phys_addr,
+MIN_MEMORY_BLOCK_SIZE * sections_per_block);
+   unlock_device_hotplug();
+   return count;
+}
+static DEVICE_ATTR(remove, S_IWUSR, NULL, memory_remove_store);
+#endif /* CONFIG_MEMORY_HOTREMOVE */
+#endif /* CONFIG_ARCH_MEMORY_PROBE */
  
  #ifdef CONFIG_MEMORY_FAILURE

  /*
@@ -790,6 +819,9 @@ bool is_memblock_offlined(struct memory_block *mem)
  static struct attribute *memory_root_attrs[] = {
  #ifdef CONFIG_ARCH_MEMORY_PROBE
_attr_probe.attr,
+#ifdef CONFIG_MEMORY_HOTREMOVE
+   _attr_remove.attr,
+#endif
  #endif
  
  #ifdef CONFIG_MEMORY_FAILURE

Re: [PATCH v2 4/5] mm: memory_hotplug: Add memory hotremove probe device

2017-11-24 Thread Andrea Reale 
Hi zhongjian,

On Fri 24 Nov 2017, 20:17, zhong jiang wrote:
> Hi, Andrea
> 
> most of server will benefit from NUMA ,it is best to sovle the issue without
> spcial restrictions.
> 
> At least we can obtain the numa information from dtb. therefore, The memory 
> can
> online correctly.

I fully agree it's an important feature, that should eventually be there. 

But, at least in my understanding, the implementation is not as
straightfoward as it looks. If I declare a memory node in the fdt, then,
at boot, the kernel will expect that memory to actually be there to be
used: this is not true if I want to plug my dimms only later at runtime.
So I think that declaring the hotpluggable memory in an fdt memory
node might not feasible without changes.

One idea could be to add a new property to memory nodes, to specify what
memory is potentially hotplugguable. For example, something like:

memory@0 {
  device_type = "memory";
  reg = <0x0 0x0 0x0 0x4000>;
  hot-add-range = <0x0 0x4000 0x0 0x4000>;
  numa-node-id=<0>;
}

memory@100 {
  device_type = "memory";
  reg = <0x100 0x0 0x0 0x4000>;
  hot-add-range = <0x100 0x4000 0x0 0x4000>;
  numa-node-id=<1>;
}

The information in this imaginary "hot-add-range" property would be
ignored at boot and only checked by the hot add process to see to which
NUMA domain some phy memory belongs.

Of course this is just an example, and my limited knowledge of fdt
doesn't make me the best person to think what's the best approach.

All this to say: in absence of a clear and agreed approach, we released
the patch with the !NUMA limitation, so that we can get early feedback.
And also in the hope to kickstart this discussion on what's the best
approach to support NUMA .

Ideas/suggestions?

Thanks,
Andrea

> 
> Thanks
> zhongjiang
> 
> On 2017/11/24 18:44, Andrea Reale wrote:
> > Hi zhongjiang,
> >
> > On Fri 24 Nov 2017, 18:35, zhong jiang wrote:
> >> HI, Andrea
> >>
> >> I don't see "memory_add_physaddr_to_nid" in arch/arm64.
> >> Am I miss something?
> > When !CONFIG_NUMA it is defined in include/linux/memory_hotplug.h as 0.
> > In patch 1/5 of this series we require !NUMA to enable
> > ARCH_ENABLE_MEMORY_HOTPLUG.
> >
> > The reason for this simplification is simply that we would not know how
> > to decide the correct node to which to add memory when NUMA is on.
> > Any suggestion on that matter is welcome. 
> >
> > Thanks,
> > Andrea
> >
> >> Thnaks
> >> zhongjiang
> >>
> >> On 2017/11/23 19:14, Andrea Reale wrote:
> >>> Adding a "remove" sysfs handle that can be used to trigger
> >>> memory hotremove manually, exactly simmetrically with
> >>> what happens with the "probe" device for hot-add.
> >>>
> >>> This is usueful for architecture that do not rely on
> >>> ACPI for memory hot-remove.
> >>>
> >>> Signed-off-by: Andrea Reale 
> >>> Signed-off-by: Maciej Bielski 
> >>> ---
> >>>  drivers/base/memory.c | 34 +-
> >>>  1 file changed, 33 insertions(+), 1 deletion(-)
> >>>
> >>> diff --git a/drivers/base/memory.c b/drivers/base/memory.c
> >>> index 1d60b58..8ccb67c 100644
> >>> --- a/drivers/base/memory.c
> >>> +++ b/drivers/base/memory.c
> >>> @@ -530,7 +530,36 @@ memory_probe_store(struct device *dev, struct 
> >>> device_attribute *attr,
> >>>  }
> >>>  
> >>>  static DEVICE_ATTR(probe, S_IWUSR, NULL, memory_probe_store);
> >>> -#endif
> >>> +
> >>> +#ifdef CONFIG_MEMORY_HOTREMOVE
> >>> +static ssize_t
> >>> +memory_remove_store(struct device *dev,
> >>> + struct device_attribute *attr, const char *buf, size_t count)
> >>> +{
> >>> + u64 phys_addr;
> >>> + int nid, ret;
> >>> + unsigned long pages_per_block = PAGES_PER_SECTION * sections_per_block;
> >>> +
> >>> + ret = kstrtoull(buf, 0, _addr);
> >>> + if (ret)
> >>> + return ret;
> >>> +
> >>> + if (phys_addr & ((pages_per_block << PAGE_SHIFT) - 1))
> >>> + return -EINVAL;
> >>> +
> >>> + nid = memory_add_physaddr_to_nid(phys_addr);
> >>> + ret = lock_device_hotplug_sysfs();
> >>> + if (ret)
> >>> + return ret;
> >>> +
> >>> + remove_memory(nid, phys_addr,
> >>> +  MIN_MEMORY_BLOCK_SIZE * sections_per_block);
> >>> + unlock_device_hotplug();
> >>> + return count;
> >>> +}
> >>> +static DEVICE_ATTR(remove, S_IWUSR, NULL, memory_remove_store);
> >>> +#endif /* CONFIG_MEMORY_HOTREMOVE */
> >>> +#endif /* CONFIG_ARCH_MEMORY_PROBE */
> >>>  
> >>>  #ifdef CONFIG_MEMORY_FAILURE
> >>>  /*
> >>> @@ -790,6 +819,9 @@ bool is_memblock_offlined(struct memory_block *mem)
> >>>  static struct attribute *memory_root_attrs[] = {
> >>>  #ifdef CONFIG_ARCH_MEMORY_PROBE
> >>>   _attr_probe.attr,
> >>> +#ifdef CONFIG_MEMORY_HOTREMOVE
> >>> + _attr_remove.attr,
> >>> +#endif
> >>>  #endif
> >>>  
> >>>  #ifdef CONFIG_MEMORY_FAILURE
> >>
> >
> > .
> >
> 
>

Re: [PATCH v2 4/5] mm: memory_hotplug: Add memory hotremove probe device

2017-11-24 Thread Andrea Reale 
Hi zhongjian,

On Fri 24 Nov 2017, 20:17, zhong jiang wrote:
> Hi, Andrea
> 
> most of server will benefit from NUMA ,it is best to sovle the issue without
> spcial restrictions.
> 
> At least we can obtain the numa information from dtb. therefore, The memory 
> can
> online correctly.

I fully agree it's an important feature, that should eventually be there. 

But, at least in my understanding, the implementation is not as
straightfoward as it looks. If I declare a memory node in the fdt, then,
at boot, the kernel will expect that memory to actually be there to be
used: this is not true if I want to plug my dimms only later at runtime.
So I think that declaring the hotpluggable memory in an fdt memory
node might not feasible without changes.

One idea could be to add a new property to memory nodes, to specify what
memory is potentially hotplugguable. For example, something like:

memory@0 {
  device_type = "memory";
  reg = <0x0 0x0 0x0 0x4000>;
  hot-add-range = <0x0 0x4000 0x0 0x4000>;
  numa-node-id=<0>;
}

memory@100 {
  device_type = "memory";
  reg = <0x100 0x0 0x0 0x4000>;
  hot-add-range = <0x100 0x4000 0x0 0x4000>;
  numa-node-id=<1>;
}

The information in this imaginary "hot-add-range" property would be
ignored at boot and only checked by the hot add process to see to which
NUMA domain some phy memory belongs.

Of course this is just an example, and my limited knowledge of fdt
doesn't make me the best person to think what's the best approach.

All this to say: in absence of a clear and agreed approach, we released
the patch with the !NUMA limitation, so that we can get early feedback.
And also in the hope to kickstart this discussion on what's the best
approach to support NUMA .

Ideas/suggestions?

Thanks,
Andrea

> 
> Thanks
> zhongjiang
> 
> On 2017/11/24 18:44, Andrea Reale wrote:
> > Hi zhongjiang,
> >
> > On Fri 24 Nov 2017, 18:35, zhong jiang wrote:
> >> HI, Andrea
> >>
> >> I don't see "memory_add_physaddr_to_nid" in arch/arm64.
> >> Am I miss something?
> > When !CONFIG_NUMA it is defined in include/linux/memory_hotplug.h as 0.
> > In patch 1/5 of this series we require !NUMA to enable
> > ARCH_ENABLE_MEMORY_HOTPLUG.
> >
> > The reason for this simplification is simply that we would not know how
> > to decide the correct node to which to add memory when NUMA is on.
> > Any suggestion on that matter is welcome. 
> >
> > Thanks,
> > Andrea
> >
> >> Thnaks
> >> zhongjiang
> >>
> >> On 2017/11/23 19:14, Andrea Reale wrote:
> >>> Adding a "remove" sysfs handle that can be used to trigger
> >>> memory hotremove manually, exactly simmetrically with
> >>> what happens with the "probe" device for hot-add.
> >>>
> >>> This is usueful for architecture that do not rely on
> >>> ACPI for memory hot-remove.
> >>>
> >>> Signed-off-by: Andrea Reale 
> >>> Signed-off-by: Maciej Bielski 
> >>> ---
> >>>  drivers/base/memory.c | 34 +-
> >>>  1 file changed, 33 insertions(+), 1 deletion(-)
> >>>
> >>> diff --git a/drivers/base/memory.c b/drivers/base/memory.c
> >>> index 1d60b58..8ccb67c 100644
> >>> --- a/drivers/base/memory.c
> >>> +++ b/drivers/base/memory.c
> >>> @@ -530,7 +530,36 @@ memory_probe_store(struct device *dev, struct 
> >>> device_attribute *attr,
> >>>  }
> >>>  
> >>>  static DEVICE_ATTR(probe, S_IWUSR, NULL, memory_probe_store);
> >>> -#endif
> >>> +
> >>> +#ifdef CONFIG_MEMORY_HOTREMOVE
> >>> +static ssize_t
> >>> +memory_remove_store(struct device *dev,
> >>> + struct device_attribute *attr, const char *buf, size_t count)
> >>> +{
> >>> + u64 phys_addr;
> >>> + int nid, ret;
> >>> + unsigned long pages_per_block = PAGES_PER_SECTION * sections_per_block;
> >>> +
> >>> + ret = kstrtoull(buf, 0, _addr);
> >>> + if (ret)
> >>> + return ret;
> >>> +
> >>> + if (phys_addr & ((pages_per_block << PAGE_SHIFT) - 1))
> >>> + return -EINVAL;
> >>> +
> >>> + nid = memory_add_physaddr_to_nid(phys_addr);
> >>> + ret = lock_device_hotplug_sysfs();
> >>> + if (ret)
> >>> + return ret;
> >>> +
> >>> + remove_memory(nid, phys_addr,
> >>> +  MIN_MEMORY_BLOCK_SIZE * sections_per_block);
> >>> + unlock_device_hotplug();
> >>> + return count;
> >>> +}
> >>> +static DEVICE_ATTR(remove, S_IWUSR, NULL, memory_remove_store);
> >>> +#endif /* CONFIG_MEMORY_HOTREMOVE */
> >>> +#endif /* CONFIG_ARCH_MEMORY_PROBE */
> >>>  
> >>>  #ifdef CONFIG_MEMORY_FAILURE
> >>>  /*
> >>> @@ -790,6 +819,9 @@ bool is_memblock_offlined(struct memory_block *mem)
> >>>  static struct attribute *memory_root_attrs[] = {
> >>>  #ifdef CONFIG_ARCH_MEMORY_PROBE
> >>>   _attr_probe.attr,
> >>> +#ifdef CONFIG_MEMORY_HOTREMOVE
> >>> + _attr_remove.attr,
> >>> +#endif
> >>>  #endif
> >>>  
> >>>  #ifdef CONFIG_MEMORY_FAILURE
> >>
> >
> > .
> >
> 
>

Re: [PATCH v2 4/5] mm: memory_hotplug: Add memory hotremove probe device

2017-11-24 Thread zhong jiang 
Hi, Andrea

most of server will benefit from NUMA ,it is best to sovle the issue without
spcial restrictions.

At least we can obtain the numa information from dtb. therefore, The memory can
online correctly.

Thanks
zhongjiang

On 2017/11/24 18:44, Andrea Reale wrote:
> Hi zhongjiang,
>
> On Fri 24 Nov 2017, 18:35, zhong jiang wrote:
>> HI, Andrea
>>
>> I don't see "memory_add_physaddr_to_nid" in arch/arm64.
>> Am I miss something?
> When !CONFIG_NUMA it is defined in include/linux/memory_hotplug.h as 0.
> In patch 1/5 of this series we require !NUMA to enable
> ARCH_ENABLE_MEMORY_HOTPLUG.
>
> The reason for this simplification is simply that we would not know how
> to decide the correct node to which to add memory when NUMA is on.
> Any suggestion on that matter is welcome. 
>
> Thanks,
> Andrea
>
>> Thnaks
>> zhongjiang
>>
>> On 2017/11/23 19:14, Andrea Reale wrote:
>>> Adding a "remove" sysfs handle that can be used to trigger
>>> memory hotremove manually, exactly simmetrically with
>>> what happens with the "probe" device for hot-add.
>>>
>>> This is usueful for architecture that do not rely on
>>> ACPI for memory hot-remove.
>>>
>>> Signed-off-by: Andrea Reale 
>>> Signed-off-by: Maciej Bielski 
>>> ---
>>>  drivers/base/memory.c | 34 +-
>>>  1 file changed, 33 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/drivers/base/memory.c b/drivers/base/memory.c
>>> index 1d60b58..8ccb67c 100644
>>> --- a/drivers/base/memory.c
>>> +++ b/drivers/base/memory.c
>>> @@ -530,7 +530,36 @@ memory_probe_store(struct device *dev, struct 
>>> device_attribute *attr,
>>>  }
>>>  
>>>  static DEVICE_ATTR(probe, S_IWUSR, NULL, memory_probe_store);
>>> -#endif
>>> +
>>> +#ifdef CONFIG_MEMORY_HOTREMOVE
>>> +static ssize_t
>>> +memory_remove_store(struct device *dev,
>>> +   struct device_attribute *attr, const char *buf, size_t count)
>>> +{
>>> +   u64 phys_addr;
>>> +   int nid, ret;
>>> +   unsigned long pages_per_block = PAGES_PER_SECTION * sections_per_block;
>>> +
>>> +   ret = kstrtoull(buf, 0, _addr);
>>> +   if (ret)
>>> +   return ret;
>>> +
>>> +   if (phys_addr & ((pages_per_block << PAGE_SHIFT) - 1))
>>> +   return -EINVAL;
>>> +
>>> +   nid = memory_add_physaddr_to_nid(phys_addr);
>>> +   ret = lock_device_hotplug_sysfs();
>>> +   if (ret)
>>> +   return ret;
>>> +
>>> +   remove_memory(nid, phys_addr,
>>> +MIN_MEMORY_BLOCK_SIZE * sections_per_block);
>>> +   unlock_device_hotplug();
>>> +   return count;
>>> +}
>>> +static DEVICE_ATTR(remove, S_IWUSR, NULL, memory_remove_store);
>>> +#endif /* CONFIG_MEMORY_HOTREMOVE */
>>> +#endif /* CONFIG_ARCH_MEMORY_PROBE */
>>>  
>>>  #ifdef CONFIG_MEMORY_FAILURE
>>>  /*
>>> @@ -790,6 +819,9 @@ bool is_memblock_offlined(struct memory_block *mem)
>>>  static struct attribute *memory_root_attrs[] = {
>>>  #ifdef CONFIG_ARCH_MEMORY_PROBE
>>> _attr_probe.attr,
>>> +#ifdef CONFIG_MEMORY_HOTREMOVE
>>> +   _attr_remove.attr,
>>> +#endif
>>>  #endif
>>>  
>>>  #ifdef CONFIG_MEMORY_FAILURE
>>
>
> .
>

Re: [PATCH v2 4/5] mm: memory_hotplug: Add memory hotremove probe device

2017-11-24 Thread zhong jiang 
Hi, Andrea

most of server will benefit from NUMA ,it is best to sovle the issue without
spcial restrictions.

At least we can obtain the numa information from dtb. therefore, The memory can
online correctly.

Thanks
zhongjiang

On 2017/11/24 18:44, Andrea Reale wrote:
> Hi zhongjiang,
>
> On Fri 24 Nov 2017, 18:35, zhong jiang wrote:
>> HI, Andrea
>>
>> I don't see "memory_add_physaddr_to_nid" in arch/arm64.
>> Am I miss something?
> When !CONFIG_NUMA it is defined in include/linux/memory_hotplug.h as 0.
> In patch 1/5 of this series we require !NUMA to enable
> ARCH_ENABLE_MEMORY_HOTPLUG.
>
> The reason for this simplification is simply that we would not know how
> to decide the correct node to which to add memory when NUMA is on.
> Any suggestion on that matter is welcome. 
>
> Thanks,
> Andrea
>
>> Thnaks
>> zhongjiang
>>
>> On 2017/11/23 19:14, Andrea Reale wrote:
>>> Adding a "remove" sysfs handle that can be used to trigger
>>> memory hotremove manually, exactly simmetrically with
>>> what happens with the "probe" device for hot-add.
>>>
>>> This is usueful for architecture that do not rely on
>>> ACPI for memory hot-remove.
>>>
>>> Signed-off-by: Andrea Reale 
>>> Signed-off-by: Maciej Bielski 
>>> ---
>>>  drivers/base/memory.c | 34 +-
>>>  1 file changed, 33 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/drivers/base/memory.c b/drivers/base/memory.c
>>> index 1d60b58..8ccb67c 100644
>>> --- a/drivers/base/memory.c
>>> +++ b/drivers/base/memory.c
>>> @@ -530,7 +530,36 @@ memory_probe_store(struct device *dev, struct 
>>> device_attribute *attr,
>>>  }
>>>  
>>>  static DEVICE_ATTR(probe, S_IWUSR, NULL, memory_probe_store);
>>> -#endif
>>> +
>>> +#ifdef CONFIG_MEMORY_HOTREMOVE
>>> +static ssize_t
>>> +memory_remove_store(struct device *dev,
>>> +   struct device_attribute *attr, const char *buf, size_t count)
>>> +{
>>> +   u64 phys_addr;
>>> +   int nid, ret;
>>> +   unsigned long pages_per_block = PAGES_PER_SECTION * sections_per_block;
>>> +
>>> +   ret = kstrtoull(buf, 0, _addr);
>>> +   if (ret)
>>> +   return ret;
>>> +
>>> +   if (phys_addr & ((pages_per_block << PAGE_SHIFT) - 1))
>>> +   return -EINVAL;
>>> +
>>> +   nid = memory_add_physaddr_to_nid(phys_addr);
>>> +   ret = lock_device_hotplug_sysfs();
>>> +   if (ret)
>>> +   return ret;
>>> +
>>> +   remove_memory(nid, phys_addr,
>>> +MIN_MEMORY_BLOCK_SIZE * sections_per_block);
>>> +   unlock_device_hotplug();
>>> +   return count;
>>> +}
>>> +static DEVICE_ATTR(remove, S_IWUSR, NULL, memory_remove_store);
>>> +#endif /* CONFIG_MEMORY_HOTREMOVE */
>>> +#endif /* CONFIG_ARCH_MEMORY_PROBE */
>>>  
>>>  #ifdef CONFIG_MEMORY_FAILURE
>>>  /*
>>> @@ -790,6 +819,9 @@ bool is_memblock_offlined(struct memory_block *mem)
>>>  static struct attribute *memory_root_attrs[] = {
>>>  #ifdef CONFIG_ARCH_MEMORY_PROBE
>>> _attr_probe.attr,
>>> +#ifdef CONFIG_MEMORY_HOTREMOVE
>>> +   _attr_remove.attr,
>>> +#endif
>>>  #endif
>>>  
>>>  #ifdef CONFIG_MEMORY_FAILURE
>>
>
> .
>

Re: [PATCH v2 4/5] mm: memory_hotplug: Add memory hotremove probe device

2017-11-24 Thread Andrea Reale 
Hi zhongjiang,

On Fri 24 Nov 2017, 18:35, zhong jiang wrote:
> HI, Andrea
> 
> I don't see "memory_add_physaddr_to_nid" in arch/arm64.
> Am I miss something?

When !CONFIG_NUMA it is defined in include/linux/memory_hotplug.h as 0.
In patch 1/5 of this series we require !NUMA to enable
ARCH_ENABLE_MEMORY_HOTPLUG.

The reason for this simplification is simply that we would not know how
to decide the correct node to which to add memory when NUMA is on.
Any suggestion on that matter is welcome. 

Thanks,
Andrea

> Thnaks
> zhongjiang
> 
> On 2017/11/23 19:14, Andrea Reale wrote:
> > Adding a "remove" sysfs handle that can be used to trigger
> > memory hotremove manually, exactly simmetrically with
> > what happens with the "probe" device for hot-add.
> >
> > This is usueful for architecture that do not rely on
> > ACPI for memory hot-remove.
> >
> > Signed-off-by: Andrea Reale 
> > Signed-off-by: Maciej Bielski 
> > ---
> >  drivers/base/memory.c | 34 +-
> >  1 file changed, 33 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/base/memory.c b/drivers/base/memory.c
> > index 1d60b58..8ccb67c 100644
> > --- a/drivers/base/memory.c
> > +++ b/drivers/base/memory.c
> > @@ -530,7 +530,36 @@ memory_probe_store(struct device *dev, struct 
> > device_attribute *attr,
> >  }
> >  
> >  static DEVICE_ATTR(probe, S_IWUSR, NULL, memory_probe_store);
> > -#endif
> > +
> > +#ifdef CONFIG_MEMORY_HOTREMOVE
> > +static ssize_t
> > +memory_remove_store(struct device *dev,
> > +   struct device_attribute *attr, const char *buf, size_t count)
> > +{
> > +   u64 phys_addr;
> > +   int nid, ret;
> > +   unsigned long pages_per_block = PAGES_PER_SECTION * sections_per_block;
> > +
> > +   ret = kstrtoull(buf, 0, _addr);
> > +   if (ret)
> > +   return ret;
> > +
> > +   if (phys_addr & ((pages_per_block << PAGE_SHIFT) - 1))
> > +   return -EINVAL;
> > +
> > +   nid = memory_add_physaddr_to_nid(phys_addr);
> > +   ret = lock_device_hotplug_sysfs();
> > +   if (ret)
> > +   return ret;
> > +
> > +   remove_memory(nid, phys_addr,
> > +MIN_MEMORY_BLOCK_SIZE * sections_per_block);
> > +   unlock_device_hotplug();
> > +   return count;
> > +}
> > +static DEVICE_ATTR(remove, S_IWUSR, NULL, memory_remove_store);
> > +#endif /* CONFIG_MEMORY_HOTREMOVE */
> > +#endif /* CONFIG_ARCH_MEMORY_PROBE */
> >  
> >  #ifdef CONFIG_MEMORY_FAILURE
> >  /*
> > @@ -790,6 +819,9 @@ bool is_memblock_offlined(struct memory_block *mem)
> >  static struct attribute *memory_root_attrs[] = {
> >  #ifdef CONFIG_ARCH_MEMORY_PROBE
> > _attr_probe.attr,
> > +#ifdef CONFIG_MEMORY_HOTREMOVE
> > +   _attr_remove.attr,
> > +#endif
> >  #endif
> >  
> >  #ifdef CONFIG_MEMORY_FAILURE
> 
>

Re: [PATCH v2 4/5] mm: memory_hotplug: Add memory hotremove probe device

2017-11-24 Thread Andrea Reale 
Hi zhongjiang,

On Fri 24 Nov 2017, 18:35, zhong jiang wrote:
> HI, Andrea
> 
> I don't see "memory_add_physaddr_to_nid" in arch/arm64.
> Am I miss something?

When !CONFIG_NUMA it is defined in include/linux/memory_hotplug.h as 0.
In patch 1/5 of this series we require !NUMA to enable
ARCH_ENABLE_MEMORY_HOTPLUG.

The reason for this simplification is simply that we would not know how
to decide the correct node to which to add memory when NUMA is on.
Any suggestion on that matter is welcome. 

Thanks,
Andrea

> Thnaks
> zhongjiang
> 
> On 2017/11/23 19:14, Andrea Reale wrote:
> > Adding a "remove" sysfs handle that can be used to trigger
> > memory hotremove manually, exactly simmetrically with
> > what happens with the "probe" device for hot-add.
> >
> > This is usueful for architecture that do not rely on
> > ACPI for memory hot-remove.
> >
> > Signed-off-by: Andrea Reale 
> > Signed-off-by: Maciej Bielski 
> > ---
> >  drivers/base/memory.c | 34 +-
> >  1 file changed, 33 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/base/memory.c b/drivers/base/memory.c
> > index 1d60b58..8ccb67c 100644
> > --- a/drivers/base/memory.c
> > +++ b/drivers/base/memory.c
> > @@ -530,7 +530,36 @@ memory_probe_store(struct device *dev, struct 
> > device_attribute *attr,
> >  }
> >  
> >  static DEVICE_ATTR(probe, S_IWUSR, NULL, memory_probe_store);
> > -#endif
> > +
> > +#ifdef CONFIG_MEMORY_HOTREMOVE
> > +static ssize_t
> > +memory_remove_store(struct device *dev,
> > +   struct device_attribute *attr, const char *buf, size_t count)
> > +{
> > +   u64 phys_addr;
> > +   int nid, ret;
> > +   unsigned long pages_per_block = PAGES_PER_SECTION * sections_per_block;
> > +
> > +   ret = kstrtoull(buf, 0, _addr);
> > +   if (ret)
> > +   return ret;
> > +
> > +   if (phys_addr & ((pages_per_block << PAGE_SHIFT) - 1))
> > +   return -EINVAL;
> > +
> > +   nid = memory_add_physaddr_to_nid(phys_addr);
> > +   ret = lock_device_hotplug_sysfs();
> > +   if (ret)
> > +   return ret;
> > +
> > +   remove_memory(nid, phys_addr,
> > +MIN_MEMORY_BLOCK_SIZE * sections_per_block);
> > +   unlock_device_hotplug();
> > +   return count;
> > +}
> > +static DEVICE_ATTR(remove, S_IWUSR, NULL, memory_remove_store);
> > +#endif /* CONFIG_MEMORY_HOTREMOVE */
> > +#endif /* CONFIG_ARCH_MEMORY_PROBE */
> >  
> >  #ifdef CONFIG_MEMORY_FAILURE
> >  /*
> > @@ -790,6 +819,9 @@ bool is_memblock_offlined(struct memory_block *mem)
> >  static struct attribute *memory_root_attrs[] = {
> >  #ifdef CONFIG_ARCH_MEMORY_PROBE
> > _attr_probe.attr,
> > +#ifdef CONFIG_MEMORY_HOTREMOVE
> > +   _attr_remove.attr,
> > +#endif
> >  #endif
> >  
> >  #ifdef CONFIG_MEMORY_FAILURE
> 
>

Re: [PATCH v2 4/5] mm: memory_hotplug: Add memory hotremove probe device

2017-11-24 Thread zhong jiang 
HI, Andrea

I don't see "memory_add_physaddr_to_nid" in arch/arm64.
Am I miss something?

Thnaks
zhongjiang

On 2017/11/23 19:14, Andrea Reale wrote:
> Adding a "remove" sysfs handle that can be used to trigger
> memory hotremove manually, exactly simmetrically with
> what happens with the "probe" device for hot-add.
>
> This is usueful for architecture that do not rely on
> ACPI for memory hot-remove.
>
> Signed-off-by: Andrea Reale 
> Signed-off-by: Maciej Bielski 
> ---
>  drivers/base/memory.c | 34 +-
>  1 file changed, 33 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/base/memory.c b/drivers/base/memory.c
> index 1d60b58..8ccb67c 100644
> --- a/drivers/base/memory.c
> +++ b/drivers/base/memory.c
> @@ -530,7 +530,36 @@ memory_probe_store(struct device *dev, struct 
> device_attribute *attr,
>  }
>  
>  static DEVICE_ATTR(probe, S_IWUSR, NULL, memory_probe_store);
> -#endif
> +
> +#ifdef CONFIG_MEMORY_HOTREMOVE
> +static ssize_t
> +memory_remove_store(struct device *dev,
> + struct device_attribute *attr, const char *buf, size_t count)
> +{
> + u64 phys_addr;
> + int nid, ret;
> + unsigned long pages_per_block = PAGES_PER_SECTION * sections_per_block;
> +
> + ret = kstrtoull(buf, 0, _addr);
> + if (ret)
> + return ret;
> +
> + if (phys_addr & ((pages_per_block << PAGE_SHIFT) - 1))
> + return -EINVAL;
> +
> + nid = memory_add_physaddr_to_nid(phys_addr);
> + ret = lock_device_hotplug_sysfs();
> + if (ret)
> + return ret;
> +
> + remove_memory(nid, phys_addr,
> +  MIN_MEMORY_BLOCK_SIZE * sections_per_block);
> + unlock_device_hotplug();
> + return count;
> +}
> +static DEVICE_ATTR(remove, S_IWUSR, NULL, memory_remove_store);
> +#endif /* CONFIG_MEMORY_HOTREMOVE */
> +#endif /* CONFIG_ARCH_MEMORY_PROBE */
>  
>  #ifdef CONFIG_MEMORY_FAILURE
>  /*
> @@ -790,6 +819,9 @@ bool is_memblock_offlined(struct memory_block *mem)
>  static struct attribute *memory_root_attrs[] = {
>  #ifdef CONFIG_ARCH_MEMORY_PROBE
>   _attr_probe.attr,
> +#ifdef CONFIG_MEMORY_HOTREMOVE
> + _attr_remove.attr,
> +#endif
>  #endif
>  
>  #ifdef CONFIG_MEMORY_FAILURE

Re: [PATCH v2 4/5] mm: memory_hotplug: Add memory hotremove probe device

2017-11-24 Thread zhong jiang 
HI, Andrea

I don't see "memory_add_physaddr_to_nid" in arch/arm64.
Am I miss something?

Thnaks
zhongjiang

On 2017/11/23 19:14, Andrea Reale wrote:
> Adding a "remove" sysfs handle that can be used to trigger
> memory hotremove manually, exactly simmetrically with
> what happens with the "probe" device for hot-add.
>
> This is usueful for architecture that do not rely on
> ACPI for memory hot-remove.
>
> Signed-off-by: Andrea Reale 
> Signed-off-by: Maciej Bielski 
> ---
>  drivers/base/memory.c | 34 +-
>  1 file changed, 33 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/base/memory.c b/drivers/base/memory.c
> index 1d60b58..8ccb67c 100644
> --- a/drivers/base/memory.c
> +++ b/drivers/base/memory.c
> @@ -530,7 +530,36 @@ memory_probe_store(struct device *dev, struct 
> device_attribute *attr,
>  }
>  
>  static DEVICE_ATTR(probe, S_IWUSR, NULL, memory_probe_store);
> -#endif
> +
> +#ifdef CONFIG_MEMORY_HOTREMOVE
> +static ssize_t
> +memory_remove_store(struct device *dev,
> + struct device_attribute *attr, const char *buf, size_t count)
> +{
> + u64 phys_addr;
> + int nid, ret;
> + unsigned long pages_per_block = PAGES_PER_SECTION * sections_per_block;
> +
> + ret = kstrtoull(buf, 0, _addr);
> + if (ret)
> + return ret;
> +
> + if (phys_addr & ((pages_per_block << PAGE_SHIFT) - 1))
> + return -EINVAL;
> +
> + nid = memory_add_physaddr_to_nid(phys_addr);
> + ret = lock_device_hotplug_sysfs();
> + if (ret)
> + return ret;
> +
> + remove_memory(nid, phys_addr,
> +  MIN_MEMORY_BLOCK_SIZE * sections_per_block);
> + unlock_device_hotplug();
> + return count;
> +}
> +static DEVICE_ATTR(remove, S_IWUSR, NULL, memory_remove_store);
> +#endif /* CONFIG_MEMORY_HOTREMOVE */
> +#endif /* CONFIG_ARCH_MEMORY_PROBE */
>  
>  #ifdef CONFIG_MEMORY_FAILURE
>  /*
> @@ -790,6 +819,9 @@ bool is_memblock_offlined(struct memory_block *mem)
>  static struct attribute *memory_root_attrs[] = {
>  #ifdef CONFIG_ARCH_MEMORY_PROBE
>   _attr_probe.attr,
> +#ifdef CONFIG_MEMORY_HOTREMOVE
> + _attr_remove.attr,
> +#endif
>  #endif
>  
>  #ifdef CONFIG_MEMORY_FAILURE

[PATCH v2 4/5] mm: memory_hotplug: Add memory hotremove probe device

2017-11-23 Thread Andrea Reale 
Adding a "remove" sysfs handle that can be used to trigger
memory hotremove manually, exactly simmetrically with
what happens with the "probe" device for hot-add.

This is usueful for architecture that do not rely on
ACPI for memory hot-remove.

Signed-off-by: Andrea Reale 
Signed-off-by: Maciej Bielski 
---
 drivers/base/memory.c | 34 +-
 1 file changed, 33 insertions(+), 1 deletion(-)

diff --git a/drivers/base/memory.c b/drivers/base/memory.c
index 1d60b58..8ccb67c 100644
--- a/drivers/base/memory.c
+++ b/drivers/base/memory.c
@@ -530,7 +530,36 @@ memory_probe_store(struct device *dev, struct 
device_attribute *attr,
 }
 
 static DEVICE_ATTR(probe, S_IWUSR, NULL, memory_probe_store);
-#endif
+
+#ifdef CONFIG_MEMORY_HOTREMOVE
+static ssize_t
+memory_remove_store(struct device *dev,
+   struct device_attribute *attr, const char *buf, size_t count)
+{
+   u64 phys_addr;
+   int nid, ret;
+   unsigned long pages_per_block = PAGES_PER_SECTION * sections_per_block;
+
+   ret = kstrtoull(buf, 0, _addr);
+   if (ret)
+   return ret;
+
+   if (phys_addr & ((pages_per_block << PAGE_SHIFT) - 1))
+   return -EINVAL;
+
+   nid = memory_add_physaddr_to_nid(phys_addr);
+   ret = lock_device_hotplug_sysfs();
+   if (ret)
+   return ret;
+
+   remove_memory(nid, phys_addr,
+MIN_MEMORY_BLOCK_SIZE * sections_per_block);
+   unlock_device_hotplug();
+   return count;
+}
+static DEVICE_ATTR(remove, S_IWUSR, NULL, memory_remove_store);
+#endif /* CONFIG_MEMORY_HOTREMOVE */
+#endif /* CONFIG_ARCH_MEMORY_PROBE */
 
 #ifdef CONFIG_MEMORY_FAILURE
 /*
@@ -790,6 +819,9 @@ bool is_memblock_offlined(struct memory_block *mem)
 static struct attribute *memory_root_attrs[] = {
 #ifdef CONFIG_ARCH_MEMORY_PROBE
_attr_probe.attr,
+#ifdef CONFIG_MEMORY_HOTREMOVE
+   _attr_remove.attr,
+#endif
 #endif
 
 #ifdef CONFIG_MEMORY_FAILURE
-- 
2.7.4

[PATCH v2 4/5] mm: memory_hotplug: Add memory hotremove probe device

2017-11-23 Thread Andrea Reale 
Adding a "remove" sysfs handle that can be used to trigger
memory hotremove manually, exactly simmetrically with
what happens with the "probe" device for hot-add.

This is usueful for architecture that do not rely on
ACPI for memory hot-remove.

Signed-off-by: Andrea Reale 
Signed-off-by: Maciej Bielski 
---
 drivers/base/memory.c | 34 +-
 1 file changed, 33 insertions(+), 1 deletion(-)

diff --git a/drivers/base/memory.c b/drivers/base/memory.c
index 1d60b58..8ccb67c 100644
--- a/drivers/base/memory.c
+++ b/drivers/base/memory.c
@@ -530,7 +530,36 @@ memory_probe_store(struct device *dev, struct 
device_attribute *attr,
 }
 
 static DEVICE_ATTR(probe, S_IWUSR, NULL, memory_probe_store);
-#endif
+
+#ifdef CONFIG_MEMORY_HOTREMOVE
+static ssize_t
+memory_remove_store(struct device *dev,
+   struct device_attribute *attr, const char *buf, size_t count)
+{
+   u64 phys_addr;
+   int nid, ret;
+   unsigned long pages_per_block = PAGES_PER_SECTION * sections_per_block;
+
+   ret = kstrtoull(buf, 0, _addr);
+   if (ret)
+   return ret;
+
+   if (phys_addr & ((pages_per_block << PAGE_SHIFT) - 1))
+   return -EINVAL;
+
+   nid = memory_add_physaddr_to_nid(phys_addr);
+   ret = lock_device_hotplug_sysfs();
+   if (ret)
+   return ret;
+
+   remove_memory(nid, phys_addr,
+MIN_MEMORY_BLOCK_SIZE * sections_per_block);
+   unlock_device_hotplug();
+   return count;
+}
+static DEVICE_ATTR(remove, S_IWUSR, NULL, memory_remove_store);
+#endif /* CONFIG_MEMORY_HOTREMOVE */
+#endif /* CONFIG_ARCH_MEMORY_PROBE */
 
 #ifdef CONFIG_MEMORY_FAILURE
 /*
@@ -790,6 +819,9 @@ bool is_memblock_offlined(struct memory_block *mem)
 static struct attribute *memory_root_attrs[] = {
 #ifdef CONFIG_ARCH_MEMORY_PROBE
_attr_probe.attr,
+#ifdef CONFIG_MEMORY_HOTREMOVE
+   _attr_remove.attr,
+#endif
 #endif
 
 #ifdef CONFIG_MEMORY_FAILURE
-- 
2.7.4