Re: [REGRESSION] [FIXED] [PATCH 1/3] mm/slab: use percpu allocator for cpu cache
On Mon, Sep 29, 2014 at 10:03:15AM -0700, Jeremiah Mahler wrote:
> Joonsoo,
>
> On Mon, Sep 29, 2014 at 04:44:18PM +0900, Joonsoo Kim wrote:
> > On Sat, Sep 27, 2014 at 11:24:49PM -0700, Jeremiah Mahler wrote:
> > > On Thu, Aug 21, 2014 at 05:11:13PM +0900, Joonsoo Kim wrote:
> > > > Because of chicken and egg problem, initializaion of SLAB is really
> > > > complicated. We need to allocate cpu cache through SLAB to make
> > > > the kmem_cache works, but, before initialization of kmem_cache,
> > > > allocation through SLAB is impossible.
> > > >
> > > > On the other hand, SLUB does initialization with more simple way. It
> > > > uses percpu allocator to allocate cpu cache so there is no chicken and
> > > > egg problem.
> > > >
> > > > So, this patch try to use percpu allocator in SLAB. This simplify
> > > > initialization step in SLAB so that we could maintain SLAB code more
> > > > easily.
> > > >
> > > > From my testing, there is no performance difference.
> > > >
> > > > Signed-off-by: Joonsoo Kim
> > >
> > > I just encountered a problem on a Lenovo Carbon X1 where it will
> > > suspend but won't resume. A bisect indicated that this patch
> > > is causing the problem.
> > >
> > > 997888488ef92da365b870247de773255227ce1f
> > >
> > > I imagine the patch author, Joonsoo Kim, might have a better idea
> > > why this is happening than I do. But if I can provide any information
> > > or run any tests that might be of help just let me know.
> >
> > Hello,
> >
> > Yeah, there is a bug. Below will fix your issue.
> > Could you test it and report the result?
> >
> > Thanks for reporting it.
> >
> > ->8---
> > From e03ed6cc554e038b86d7b3a72b89d94e9ea808ba Mon Sep 17 00:00:00 2001
> > From: Joonsoo Kim
> > Date: Mon, 29 Sep 2014 16:30:43 +0900
> > Subject: [PATCH] mm/slab: fix cpu on/off handling
> >
> > When cpu off, we flush all cpu cached objects to it's own slab.
> > free_block() is used for this purpose and it's role is just to flush
> > objects from array_cache to proper slab. It doesn't adjust array_cache's
> > internal fields so we should manually reset them to proper value.
> > Without this fix, we maintain free objects duplicately, one is in
> > cpu cache, and, the other one is in the slab. So system would be broken.
> >
> > Reported-by: Jeremiah Mahler
> > Signed-off-by: Joonsoo Kim
> > ---
> > mm/slab.c |4 +++-
> > 1 file changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/mm/slab.c b/mm/slab.c
> > index 1162f0e..ce289b4 100644
> > --- a/mm/slab.c
> > +++ b/mm/slab.c
> > @@ -1102,8 +1102,10 @@ static void cpuup_canceled(long cpu)
> >
> > /* cpu is dead; no one can alloc from it. */
> > nc = per_cpu_ptr(cachep->cpu_cache, cpu);
> > - if (nc)
> > + if (nc) {
> > free_block(cachep, nc->entry, nc->avail, node, &list);
> > + nc->avail = 0;
> > + }
> >
> > if (!cpumask_empty(mask)) {
> > spin_unlock_irq(&n->list_lock);
> > --
> > 1.7.9.5
> >
>
> That fixed the problem. Thanks!
>
> Tested-by: Jeremiah Mahler
Good!
Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [REGRESSION] [FIXED] [PATCH 1/3] mm/slab: use percpu allocator for cpu cache
Joonsoo,
On Mon, Sep 29, 2014 at 04:44:18PM +0900, Joonsoo Kim wrote:
> On Sat, Sep 27, 2014 at 11:24:49PM -0700, Jeremiah Mahler wrote:
> > On Thu, Aug 21, 2014 at 05:11:13PM +0900, Joonsoo Kim wrote:
> > > Because of chicken and egg problem, initializaion of SLAB is really
> > > complicated. We need to allocate cpu cache through SLAB to make
> > > the kmem_cache works, but, before initialization of kmem_cache,
> > > allocation through SLAB is impossible.
> > >
> > > On the other hand, SLUB does initialization with more simple way. It
> > > uses percpu allocator to allocate cpu cache so there is no chicken and
> > > egg problem.
> > >
> > > So, this patch try to use percpu allocator in SLAB. This simplify
> > > initialization step in SLAB so that we could maintain SLAB code more
> > > easily.
> > >
> > > From my testing, there is no performance difference.
> > >
> > > Signed-off-by: Joonsoo Kim
> >
> > I just encountered a problem on a Lenovo Carbon X1 where it will
> > suspend but won't resume. A bisect indicated that this patch
> > is causing the problem.
> >
> > 997888488ef92da365b870247de773255227ce1f
> >
> > I imagine the patch author, Joonsoo Kim, might have a better idea
> > why this is happening than I do. But if I can provide any information
> > or run any tests that might be of help just let me know.
>
> Hello,
>
> Yeah, there is a bug. Below will fix your issue.
> Could you test it and report the result?
>
> Thanks for reporting it.
>
> ->8---
> From e03ed6cc554e038b86d7b3a72b89d94e9ea808ba Mon Sep 17 00:00:00 2001
> From: Joonsoo Kim
> Date: Mon, 29 Sep 2014 16:30:43 +0900
> Subject: [PATCH] mm/slab: fix cpu on/off handling
>
> When cpu off, we flush all cpu cached objects to it's own slab.
> free_block() is used for this purpose and it's role is just to flush
> objects from array_cache to proper slab. It doesn't adjust array_cache's
> internal fields so we should manually reset them to proper value.
> Without this fix, we maintain free objects duplicately, one is in
> cpu cache, and, the other one is in the slab. So system would be broken.
>
> Reported-by: Jeremiah Mahler
> Signed-off-by: Joonsoo Kim
> ---
> mm/slab.c |4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/mm/slab.c b/mm/slab.c
> index 1162f0e..ce289b4 100644
> --- a/mm/slab.c
> +++ b/mm/slab.c
> @@ -1102,8 +1102,10 @@ static void cpuup_canceled(long cpu)
>
> /* cpu is dead; no one can alloc from it. */
> nc = per_cpu_ptr(cachep->cpu_cache, cpu);
> - if (nc)
> + if (nc) {
> free_block(cachep, nc->entry, nc->avail, node, &list);
> + nc->avail = 0;
> + }
>
> if (!cpumask_empty(mask)) {
> spin_unlock_irq(&n->list_lock);
> --
> 1.7.9.5
>
That fixed the problem. Thanks!
Tested-by: Jeremiah Mahler
--
Jeremiah Mahler
jmmah...@gmail.com
http://github.com/jmahler
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [REGRESSION] [PATCH 1/3] mm/slab: use percpu allocator for cpu cache
On Sat, Sep 27, 2014 at 11:24:49PM -0700, Jeremiah Mahler wrote:
> On Thu, Aug 21, 2014 at 05:11:13PM +0900, Joonsoo Kim wrote:
> > Because of chicken and egg problem, initializaion of SLAB is really
> > complicated. We need to allocate cpu cache through SLAB to make
> > the kmem_cache works, but, before initialization of kmem_cache,
> > allocation through SLAB is impossible.
> >
> > On the other hand, SLUB does initialization with more simple way. It
> > uses percpu allocator to allocate cpu cache so there is no chicken and
> > egg problem.
> >
> > So, this patch try to use percpu allocator in SLAB. This simplify
> > initialization step in SLAB so that we could maintain SLAB code more
> > easily.
> >
> > From my testing, there is no performance difference.
> >
> > Signed-off-by: Joonsoo Kim
>
> I just encountered a problem on a Lenovo Carbon X1 where it will
> suspend but won't resume. A bisect indicated that this patch
> is causing the problem.
>
> 997888488ef92da365b870247de773255227ce1f
>
> I imagine the patch author, Joonsoo Kim, might have a better idea
> why this is happening than I do. But if I can provide any information
> or run any tests that might be of help just let me know.
Hello,
Yeah, there is a bug. Below will fix your issue.
Could you test it and report the result?
Thanks for reporting it.
->8---
>From e03ed6cc554e038b86d7b3a72b89d94e9ea808ba Mon Sep 17 00:00:00 2001
From: Joonsoo Kim
Date: Mon, 29 Sep 2014 16:30:43 +0900
Subject: [PATCH] mm/slab: fix cpu on/off handling
When cpu off, we flush all cpu cached objects to it's own slab.
free_block() is used for this purpose and it's role is just to flush
objects from array_cache to proper slab. It doesn't adjust array_cache's
internal fields so we should manually reset them to proper value.
Without this fix, we maintain free objects duplicately, one is in
cpu cache, and, the other one is in the slab. So system would be broken.
Reported-by: Jeremiah Mahler
Signed-off-by: Joonsoo Kim
---
mm/slab.c |4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/mm/slab.c b/mm/slab.c
index 1162f0e..ce289b4 100644
--- a/mm/slab.c
+++ b/mm/slab.c
@@ -1102,8 +1102,10 @@ static void cpuup_canceled(long cpu)
/* cpu is dead; no one can alloc from it. */
nc = per_cpu_ptr(cachep->cpu_cache, cpu);
- if (nc)
+ if (nc) {
free_block(cachep, nc->entry, nc->avail, node, &list);
+ nc->avail = 0;
+ }
if (!cpumask_empty(mask)) {
spin_unlock_irq(&n->list_lock);
--
1.7.9.5
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [REGRESSION] [PATCH 1/3] mm/slab: use percpu allocator for cpu cache
On Sun, Sep 28, 2014 at 11:38:51AM -0500, Christoph Lameter wrote: > On Sat, 27 Sep 2014, Jeremiah Mahler wrote: > > > I just encountered a problem on a Lenovo Carbon X1 where it will > > suspend but won't resume. A bisect indicated that this patch > > is causing the problem. > > Could you please not quote the whole patch. Took me a while to find what > you were saying. > Sorry about that. > > 997888488ef92da365b870247de773255227ce1f > > > > I imagine the patch author, Joonsoo Kim, might have a better idea > > why this is happening than I do. But if I can provide any information > > or run any tests that might be of help just let me know. > > Could you provide more details? Any messages when the system is trying to > resume? > When I press Fn to resume there is a very brief flicker of the wireless indicator light, as if it is trying to resume, but then it remains suspended. There are no messages on the screen or anything else. -- Jeremiah Mahler jmmah...@gmail.com http://github.com/jmahler -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [REGRESSION] [PATCH 1/3] mm/slab: use percpu allocator for cpu cache
On Sat, 27 Sep 2014, Jeremiah Mahler wrote: > I just encountered a problem on a Lenovo Carbon X1 where it will > suspend but won't resume. A bisect indicated that this patch > is causing the problem. Could you please not quote the whole patch. Took me a while to find what you were saying. > 997888488ef92da365b870247de773255227ce1f > > I imagine the patch author, Joonsoo Kim, might have a better idea > why this is happening than I do. But if I can provide any information > or run any tests that might be of help just let me know. Could you provide more details? Any messages when the system is trying to resume? -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [REGRESSION] [PATCH 1/3] mm/slab: use percpu allocator for cpu cache
On Thu, Aug 21, 2014 at 05:11:13PM +0900, Joonsoo Kim wrote:
> Because of chicken and egg problem, initializaion of SLAB is really
> complicated. We need to allocate cpu cache through SLAB to make
> the kmem_cache works, but, before initialization of kmem_cache,
> allocation through SLAB is impossible.
>
> On the other hand, SLUB does initialization with more simple way. It
> uses percpu allocator to allocate cpu cache so there is no chicken and
> egg problem.
>
> So, this patch try to use percpu allocator in SLAB. This simplify
> initialization step in SLAB so that we could maintain SLAB code more
> easily.
>
> From my testing, there is no performance difference.
>
> Signed-off-by: Joonsoo Kim
> ---
> include/linux/slab_def.h | 20 +---
> mm/slab.c| 237
> +++---
> mm/slab.h|1 -
> 3 files changed, 81 insertions(+), 177 deletions(-)
>
> diff --git a/include/linux/slab_def.h b/include/linux/slab_def.h
> index 8235dfb..b869d16 100644
> --- a/include/linux/slab_def.h
> +++ b/include/linux/slab_def.h
> @@ -8,6 +8,8 @@
> */
>
> struct kmem_cache {
> + struct array_cache __percpu *cpu_cache;
> +
> /* 1) Cache tunables. Protected by slab_mutex */
> unsigned int batchcount;
> unsigned int limit;
> @@ -71,23 +73,7 @@ struct kmem_cache {
> struct memcg_cache_params *memcg_params;
> #endif
>
> -/* 6) per-cpu/per-node data, touched during every alloc/free */
> - /*
> - * We put array[] at the end of kmem_cache, because we want to size
> - * this array to nr_cpu_ids slots instead of NR_CPUS
> - * (see kmem_cache_init())
> - * We still use [NR_CPUS] and not [1] or [0] because cache_cache
> - * is statically defined, so we reserve the max number of cpus.
> - *
> - * We also need to guarantee that the list is able to accomodate a
> - * pointer for each node since "nodelists" uses the remainder of
> - * available pointers.
> - */
> - struct kmem_cache_node **node;
> - struct array_cache *array[NR_CPUS + MAX_NUMNODES];
> - /*
> - * Do not add fields after array[]
> - */
> + struct kmem_cache_node *node[MAX_NUMNODES];
> };
>
> #endif /* _LINUX_SLAB_DEF_H */
> diff --git a/mm/slab.c b/mm/slab.c
> index 5927a17..09b060e 100644
> --- a/mm/slab.c
> +++ b/mm/slab.c
> @@ -237,11 +237,10 @@ struct arraycache_init {
> /*
> * Need this for bootstrapping a per node allocator.
> */
> -#define NUM_INIT_LISTS (3 * MAX_NUMNODES)
> +#define NUM_INIT_LISTS (2 * MAX_NUMNODES)
> static struct kmem_cache_node __initdata
> init_kmem_cache_node[NUM_INIT_LISTS];
> #define CACHE_CACHE 0
> -#define SIZE_AC MAX_NUMNODES
> -#define SIZE_NODE (2 * MAX_NUMNODES)
> +#define SIZE_NODE (MAX_NUMNODES)
>
> static int drain_freelist(struct kmem_cache *cache,
> struct kmem_cache_node *n, int tofree);
> @@ -253,7 +252,6 @@ static void cache_reap(struct work_struct *unused);
>
> static int slab_early_init = 1;
>
> -#define INDEX_AC kmalloc_index(sizeof(struct arraycache_init))
> #define INDEX_NODE kmalloc_index(sizeof(struct kmem_cache_node))
>
> static void kmem_cache_node_init(struct kmem_cache_node *parent)
> @@ -458,9 +456,6 @@ static inline unsigned int obj_to_index(const struct
> kmem_cache *cache,
> return reciprocal_divide(offset, cache->reciprocal_buffer_size);
> }
>
> -static struct arraycache_init initarray_generic =
> -{ {0, BOOT_CPUCACHE_ENTRIES, 1, 0} };
> -
> /* internal cache of cache description objs */
> static struct kmem_cache kmem_cache_boot = {
> .batchcount = 1,
> @@ -476,7 +471,7 @@ static DEFINE_PER_CPU(struct delayed_work,
> slab_reap_work);
>
> static inline struct array_cache *cpu_cache_get(struct kmem_cache *cachep)
> {
> - return cachep->array[smp_processor_id()];
> + return this_cpu_ptr(cachep->cpu_cache);
> }
>
> static size_t calculate_freelist_size(int nr_objs, size_t align)
> @@ -1096,9 +1091,6 @@ static void cpuup_canceled(long cpu)
> struct alien_cache **alien;
> LIST_HEAD(list);
>
> - /* cpu is dead; no one can alloc from it. */
> - nc = cachep->array[cpu];
> - cachep->array[cpu] = NULL;
> n = get_node(cachep, node);
>
> if (!n)
> @@ -1108,6 +1100,9 @@ static void cpuup_canceled(long cpu)
>
> /* Free limit for this kmem_cache_node */
> n->free_limit -= cachep->batchcount;
> +
> + /* cpu is dead; no one can alloc from it. */
> + nc = per_cpu_ptr(cachep->cpu_cache, cpu);
> if (nc)
> free_block(cachep, nc->entry, nc->avail, node, &list);
>
> @@ -1135,7 +1130,6 @@ static void cpuup_canceled(long cpu)
> }
> free_array_cache:
> slabs_destroy(cachep, &list);
> - kfree(nc);
> }
> /*
>
Re: [PATCH 1/3] mm/slab: use percpu allocator for cpu cache
On Wed, Aug 27, 2014 at 06:37:33PM -0500, Christoph Lameter wrote:
> One minor nit. Otherwise
>
> Acked-by: Christoph Lameter
>
> On Thu, 21 Aug 2014, Joonsoo Kim wrote:
>
> > @@ -2041,56 +1982,63 @@ static size_t calculate_slab_order(struct
> > kmem_cache *cachep,
> > return left_over;
> > }
> >
> > +static int alloc_kmem_cache_cpus(struct kmem_cache *cachep, int entries,
> > + int batchcount)
> > +{
> > + cachep->cpu_cache = __alloc_kmem_cache_cpus(cachep, entries,
> > + batchcount);
> > + if (!cachep->cpu_cache)
> > + return 1;
> > +
> > + return 0;
> > +}
>
> Do we really need this trivial function? It doesnt do anything useful as
> far as I can tell.
Hello,
You are right. I will remove it in next spin.
Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 1/3] mm/slab: use percpu allocator for cpu cache
One minor nit. Otherwise
Acked-by: Christoph Lameter
On Thu, 21 Aug 2014, Joonsoo Kim wrote:
> @@ -2041,56 +1982,63 @@ static size_t calculate_slab_order(struct kmem_cache
> *cachep,
> return left_over;
> }
>
> +static int alloc_kmem_cache_cpus(struct kmem_cache *cachep, int entries,
> + int batchcount)
> +{
> + cachep->cpu_cache = __alloc_kmem_cache_cpus(cachep, entries,
> + batchcount);
> + if (!cachep->cpu_cache)
> + return 1;
> +
> + return 0;
> +}
Do we really need this trivial function? It doesnt do anything useful as
far as I can tell.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 1/3] mm/slab: use percpu allocator for cpu cache
On Tue, 26 Aug 2014, Joonsoo Kim wrote: > > What case? SLUB uses a linked list and therefore does not have these > > storage requirements. > > I misunderstand that you mentioned just memory usage. My *any case* > means memory usage of previous SLAB and SLAB with this percpu alloc > change. Sorry for confusion. Ok. True the total amount of memory used does not increase. > > > I know that percpu allocator occupy vmalloc space, so maybe we could > > > exhaust vmalloc space on 32 bit. 64 bit has no problem on it. > > > How many cores does largest 32 bit system have? Is it possible > > > to exhaust vmalloc space if we use percpu allocator? > > > > There were NUMA systems on x86 a while back (not sure if they still > > exists) with 128 or so processors. > > > > Some people boot 32 bit kernels on contemporary servers. The Intel ones > > max out at 18 cores (36 hyperthreaded). I think they support up to 8 > > scokets. So 8 * 36? > > > > > > Its different on other platforms with much higher numbers. Power can > > easily go up to hundreds of hardware threads and SGI Altixes 7 yearsago > > where at 8000 or so. > > Okay... These large systems with 32 bit kernel could be break with this > change. I will do more investigation. Possibly, I will drop this patch. :) Wait the last system mentioned are 64 bit. SGI definitely. Power probably too. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 1/3] mm/slab: use percpu allocator for cpu cache
On Mon, Aug 25, 2014 at 08:13:58AM -0500, Christoph Lameter wrote: > On Mon, 25 Aug 2014, Joonsoo Kim wrote: > > > On Thu, Aug 21, 2014 at 09:21:30AM -0500, Christoph Lameter wrote: > > > On Thu, 21 Aug 2014, Joonsoo Kim wrote: > > > > > > > So, this patch try to use percpu allocator in SLAB. This simplify > > > > initialization step in SLAB so that we could maintain SLAB code more > > > > easily. > > > > > > I thought about this a couple of times but the amount of memory used for > > > the per cpu arrays can be huge. In contrast to slub which needs just a > > > few pointers, slab requires one pointer per object that can be in the > > > local cache. CC Tj. > > > > > > Lets say we have 300 caches and we allow 1000 objects to be cached per > > > cpu. That is 300k pointers per cpu. 1.2M on 32 bit. 2.4M per cpu on > > > 64bit. > > > > Amount of memory we need to keep pointers for object is same in any case. > > What case? SLUB uses a linked list and therefore does not have these > storage requirements. I misunderstand that you mentioned just memory usage. My *any case* means memory usage of previous SLAB and SLAB with this percpu alloc change. Sorry for confusion. > > > I know that percpu allocator occupy vmalloc space, so maybe we could > > exhaust vmalloc space on 32 bit. 64 bit has no problem on it. > > How many cores does largest 32 bit system have? Is it possible > > to exhaust vmalloc space if we use percpu allocator? > > There were NUMA systems on x86 a while back (not sure if they still > exists) with 128 or so processors. > > Some people boot 32 bit kernels on contemporary servers. The Intel ones > max out at 18 cores (36 hyperthreaded). I think they support up to 8 > scokets. So 8 * 36? > > > Its different on other platforms with much higher numbers. Power can > easily go up to hundreds of hardware threads and SGI Altixes 7 yearsago > where at 8000 or so. Okay... These large systems with 32 bit kernel could be break with this change. I will do more investigation. Possibly, I will drop this patch. :) Thanks. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 1/3] mm/slab: use percpu allocator for cpu cache
On Mon, 25 Aug 2014, Joonsoo Kim wrote: > On Thu, Aug 21, 2014 at 09:21:30AM -0500, Christoph Lameter wrote: > > On Thu, 21 Aug 2014, Joonsoo Kim wrote: > > > > > So, this patch try to use percpu allocator in SLAB. This simplify > > > initialization step in SLAB so that we could maintain SLAB code more > > > easily. > > > > I thought about this a couple of times but the amount of memory used for > > the per cpu arrays can be huge. In contrast to slub which needs just a > > few pointers, slab requires one pointer per object that can be in the > > local cache. CC Tj. > > > > Lets say we have 300 caches and we allow 1000 objects to be cached per > > cpu. That is 300k pointers per cpu. 1.2M on 32 bit. 2.4M per cpu on > > 64bit. > > Amount of memory we need to keep pointers for object is same in any case. What case? SLUB uses a linked list and therefore does not have these storage requirements. > I know that percpu allocator occupy vmalloc space, so maybe we could > exhaust vmalloc space on 32 bit. 64 bit has no problem on it. > How many cores does largest 32 bit system have? Is it possible > to exhaust vmalloc space if we use percpu allocator? There were NUMA systems on x86 a while back (not sure if they still exists) with 128 or so processors. Some people boot 32 bit kernels on contemporary servers. The Intel ones max out at 18 cores (36 hyperthreaded). I think they support up to 8 scokets. So 8 * 36? Its different on other platforms with much higher numbers. Power can easily go up to hundreds of hardware threads and SGI Altixes 7 yearsago where at 8000 or so. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 1/3] mm/slab: use percpu allocator for cpu cache
On Thu, Aug 21, 2014 at 09:21:30AM -0500, Christoph Lameter wrote: > On Thu, 21 Aug 2014, Joonsoo Kim wrote: > > > So, this patch try to use percpu allocator in SLAB. This simplify > > initialization step in SLAB so that we could maintain SLAB code more > > easily. > > I thought about this a couple of times but the amount of memory used for > the per cpu arrays can be huge. In contrast to slub which needs just a > few pointers, slab requires one pointer per object that can be in the > local cache. CC Tj. > > Lets say we have 300 caches and we allow 1000 objects to be cached per > cpu. That is 300k pointers per cpu. 1.2M on 32 bit. 2.4M per cpu on > 64bit. Hello, Christoph. Amount of memory we need to keep pointers for object is same in any case. I know that percpu allocator occupy vmalloc space, so maybe we could exhaust vmalloc space on 32 bit. 64 bit has no problem on it. How many cores does largest 32 bit system have? Is it possible to exhaust vmalloc space if we use percpu allocator? Thanks. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 1/3] mm/slab: use percpu allocator for cpu cache
On Thu, 21 Aug 2014, Joonsoo Kim wrote: > So, this patch try to use percpu allocator in SLAB. This simplify > initialization step in SLAB so that we could maintain SLAB code more > easily. I thought about this a couple of times but the amount of memory used for the per cpu arrays can be huge. In contrast to slub which needs just a few pointers, slab requires one pointer per object that can be in the local cache. CC Tj. Lets say we have 300 caches and we allow 1000 objects to be cached per cpu. That is 300k pointers per cpu. 1.2M on 32 bit. 2.4M per cpu on 64bit. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH 1/3] mm/slab: use percpu allocator for cpu cache
Because of chicken and egg problem, initializaion of SLAB is really
complicated. We need to allocate cpu cache through SLAB to make
the kmem_cache works, but, before initialization of kmem_cache,
allocation through SLAB is impossible.
On the other hand, SLUB does initialization with more simple way. It
uses percpu allocator to allocate cpu cache so there is no chicken and
egg problem.
So, this patch try to use percpu allocator in SLAB. This simplify
initialization step in SLAB so that we could maintain SLAB code more
easily.
>From my testing, there is no performance difference.
Signed-off-by: Joonsoo Kim
---
include/linux/slab_def.h | 20 +---
mm/slab.c| 237 +++---
mm/slab.h|1 -
3 files changed, 81 insertions(+), 177 deletions(-)
diff --git a/include/linux/slab_def.h b/include/linux/slab_def.h
index 8235dfb..b869d16 100644
--- a/include/linux/slab_def.h
+++ b/include/linux/slab_def.h
@@ -8,6 +8,8 @@
*/
struct kmem_cache {
+ struct array_cache __percpu *cpu_cache;
+
/* 1) Cache tunables. Protected by slab_mutex */
unsigned int batchcount;
unsigned int limit;
@@ -71,23 +73,7 @@ struct kmem_cache {
struct memcg_cache_params *memcg_params;
#endif
-/* 6) per-cpu/per-node data, touched during every alloc/free */
- /*
-* We put array[] at the end of kmem_cache, because we want to size
-* this array to nr_cpu_ids slots instead of NR_CPUS
-* (see kmem_cache_init())
-* We still use [NR_CPUS] and not [1] or [0] because cache_cache
-* is statically defined, so we reserve the max number of cpus.
-*
-* We also need to guarantee that the list is able to accomodate a
-* pointer for each node since "nodelists" uses the remainder of
-* available pointers.
-*/
- struct kmem_cache_node **node;
- struct array_cache *array[NR_CPUS + MAX_NUMNODES];
- /*
-* Do not add fields after array[]
-*/
+ struct kmem_cache_node *node[MAX_NUMNODES];
};
#endif /* _LINUX_SLAB_DEF_H */
diff --git a/mm/slab.c b/mm/slab.c
index 5927a17..09b060e 100644
--- a/mm/slab.c
+++ b/mm/slab.c
@@ -237,11 +237,10 @@ struct arraycache_init {
/*
* Need this for bootstrapping a per node allocator.
*/
-#define NUM_INIT_LISTS (3 * MAX_NUMNODES)
+#define NUM_INIT_LISTS (2 * MAX_NUMNODES)
static struct kmem_cache_node __initdata init_kmem_cache_node[NUM_INIT_LISTS];
#defineCACHE_CACHE 0
-#defineSIZE_AC MAX_NUMNODES
-#defineSIZE_NODE (2 * MAX_NUMNODES)
+#defineSIZE_NODE (MAX_NUMNODES)
static int drain_freelist(struct kmem_cache *cache,
struct kmem_cache_node *n, int tofree);
@@ -253,7 +252,6 @@ static void cache_reap(struct work_struct *unused);
static int slab_early_init = 1;
-#define INDEX_AC kmalloc_index(sizeof(struct arraycache_init))
#define INDEX_NODE kmalloc_index(sizeof(struct kmem_cache_node))
static void kmem_cache_node_init(struct kmem_cache_node *parent)
@@ -458,9 +456,6 @@ static inline unsigned int obj_to_index(const struct
kmem_cache *cache,
return reciprocal_divide(offset, cache->reciprocal_buffer_size);
}
-static struct arraycache_init initarray_generic =
-{ {0, BOOT_CPUCACHE_ENTRIES, 1, 0} };
-
/* internal cache of cache description objs */
static struct kmem_cache kmem_cache_boot = {
.batchcount = 1,
@@ -476,7 +471,7 @@ static DEFINE_PER_CPU(struct delayed_work, slab_reap_work);
static inline struct array_cache *cpu_cache_get(struct kmem_cache *cachep)
{
- return cachep->array[smp_processor_id()];
+ return this_cpu_ptr(cachep->cpu_cache);
}
static size_t calculate_freelist_size(int nr_objs, size_t align)
@@ -1096,9 +1091,6 @@ static void cpuup_canceled(long cpu)
struct alien_cache **alien;
LIST_HEAD(list);
- /* cpu is dead; no one can alloc from it. */
- nc = cachep->array[cpu];
- cachep->array[cpu] = NULL;
n = get_node(cachep, node);
if (!n)
@@ -1108,6 +1100,9 @@ static void cpuup_canceled(long cpu)
/* Free limit for this kmem_cache_node */
n->free_limit -= cachep->batchcount;
+
+ /* cpu is dead; no one can alloc from it. */
+ nc = per_cpu_ptr(cachep->cpu_cache, cpu);
if (nc)
free_block(cachep, nc->entry, nc->avail, node, &list);
@@ -1135,7 +1130,6 @@ static void cpuup_canceled(long cpu)
}
free_array_cache:
slabs_destroy(cachep, &list);
- kfree(nc);
}
/*
* In the previous loop, all the objects were freed to
@@ -1172,32 +1166,23 @@ static int cpuup_prepare(long cpu)
* array caches
*/
list_for_each_entry(cachep, &slab_caches, list) {
- struct array_cache *n
