Re: [PATCH v6 00/29] kmem controller for memcg.
On 11/02/2012 08:25 PM, JoonSoo Kim wrote: > Hello, Glauber. > > 2012/11/2 Glauber Costa : >> On 11/02/2012 04:04 AM, Andrew Morton wrote: >>> On Thu, 1 Nov 2012 16:07:16 +0400 >>> Glauber Costa wrote: >>> Hi, This work introduces the kernel memory controller for memcg. Unlike previous submissions, this includes the whole controller, comprised of slab and stack memory. >>> >>> I'm in the middle of (re)reading all this. Meanwhile I'll push it all >>> out to http://ozlabs.org/~akpm/mmots/ for the crazier testers. >>> >>> One thing: >>> Numbers can be found at https://lkml.org/lkml/2012/9/13/239 >>> >>> You claim in the above that the fork worload is 'slab intensive". Or >>> at least, you seem to - it's a bit fuzzy. >>> >>> But how slab intensive is it, really? >>> >>> What is extremely slab intensive is networking. The networking guys >>> are very sensitive to slab performance. If this hasn't already been >>> done, could you please determine what impact this has upon networking? >>> I expect Eric Dumazet, Dave Miller and Tom Herbert could suggest >>> testing approaches. >>> >> >> I can test it, but unfortunately I am unlikely to get to prepare a good >> environment before Barcelona. >> >> I know, however, that Greg Thelen was testing netperf in his setup. >> Greg, do you have any publishable numbers you could share? > > Below is my humble opinion. > I am worrying about data cache footprint which is possibly caused by > this patchset, especially slab implementation. > If there are several memcg cgroups, each cgroup has it's own kmem_caches. I answered the performance part in response to Tejun's response. Let me just add something here: Just keep in mind this is not "per memcg", this is "per memcg that are kernel-memory limited". So in a sense, you are only paying this, and allocate from different caches, if you runtime enable this. This should all be documented in the Documentation/ patch. But let me know if there is anything that needs further clarification -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v6 00/29] kmem controller for memcg.
On 11/03/2012 12:06 AM, Tejun Heo wrote: > Hey, Joonsoo. > > On Sat, Nov 03, 2012 at 04:25:59AM +0900, JoonSoo Kim wrote: >> I am worrying about data cache footprint which is possibly caused by >> this patchset, especially slab implementation. >> If there are several memcg cgroups, each cgroup has it's own kmem_caches. >> When each group do slab-intensive job hard, data cache may be overflowed >> easily, >> and cache miss rate will be high, therefore this would decrease system >> performance highly. > > It would be nice to be able to remove such overhead too, but the > baselines for cgroup implementations (well, at least the ones that I > think important) in somewhat decreasing priority are... > > 1. Don't over-complicate the target subsystem. > > 2. Overhead when cgroup is not used should be minimal. Prefereably to >the level of being unnoticeable. > > 3. Overhead while cgroup is being actively used should be reasonable. > > If you wanna split your system into N groups and maintain memory > resource segregation among them, I don't think it's unreasonable to > ask for paying data cache footprint overhead. > > So, while improvements would be nice, I wouldn't consider overheads of > this type as a blocker. > > Thanks. > There is another thing I should add. We are essentially replicating all the allocator meta-data, so if you look at it, this is exactly the same thing as workloads that allocate from different allocators (i.e.: a lot of network structures, and a lot of dentries). In this sense, it really basically depends what is your comparison point. Full containers - the main (but not exclusive) reason for this, are more or less an alternative for virtual machines. In those, you would be allocating from a different cache because you would be getting those through a bunch of memory address translations. From this, we do a lot better, since we only change the cache you allocate from, keeping all the rest unchanged. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v6 00/29] kmem controller for memcg.
On Fri, Nov 2, 2012 at 12:41 AM, Glauber Costa wrote: > On 11/02/2012 04:04 AM, Andrew Morton wrote: >> On Thu, 1 Nov 2012 16:07:16 +0400 >> Glauber Costa wrote: >> >>> Hi, >>> >>> This work introduces the kernel memory controller for memcg. Unlike previous >>> submissions, this includes the whole controller, comprised of slab and stack >>> memory. >> >> I'm in the middle of (re)reading all this. Meanwhile I'll push it all >> out to http://ozlabs.org/~akpm/mmots/ for the crazier testers. >> >> One thing: >> >>> Numbers can be found at https://lkml.org/lkml/2012/9/13/239 >> >> You claim in the above that the fork worload is 'slab intensive". Or >> at least, you seem to - it's a bit fuzzy. >> >> But how slab intensive is it, really? >> >> What is extremely slab intensive is networking. The networking guys >> are very sensitive to slab performance. If this hasn't already been >> done, could you please determine what impact this has upon networking? >> I expect Eric Dumazet, Dave Miller and Tom Herbert could suggest >> testing approaches. >> > > I can test it, but unfortunately I am unlikely to get to prepare a good > environment before Barcelona. > > I know, however, that Greg Thelen was testing netperf in his setup. > Greg, do you have any publishable numbers you could share? I should have some netperf numbers next week. Sorry I've been distracted by other projects recently. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v6 00/29] kmem controller for memcg.
Hey, Joonsoo. On Sat, Nov 03, 2012 at 04:25:59AM +0900, JoonSoo Kim wrote: > I am worrying about data cache footprint which is possibly caused by > this patchset, especially slab implementation. > If there are several memcg cgroups, each cgroup has it's own kmem_caches. > When each group do slab-intensive job hard, data cache may be overflowed > easily, > and cache miss rate will be high, therefore this would decrease system > performance highly. It would be nice to be able to remove such overhead too, but the baselines for cgroup implementations (well, at least the ones that I think important) in somewhat decreasing priority are... 1. Don't over-complicate the target subsystem. 2. Overhead when cgroup is not used should be minimal. Prefereably to the level of being unnoticeable. 3. Overhead while cgroup is being actively used should be reasonable. If you wanna split your system into N groups and maintain memory resource segregation among them, I don't think it's unreasonable to ask for paying data cache footprint overhead. So, while improvements would be nice, I wouldn't consider overheads of this type as a blocker. Thanks. -- tejun -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v6 00/29] kmem controller for memcg.
Hello, Glauber. 2012/11/2 Glauber Costa : > On 11/02/2012 04:04 AM, Andrew Morton wrote: >> On Thu, 1 Nov 2012 16:07:16 +0400 >> Glauber Costa wrote: >> >>> Hi, >>> >>> This work introduces the kernel memory controller for memcg. Unlike previous >>> submissions, this includes the whole controller, comprised of slab and stack >>> memory. >> >> I'm in the middle of (re)reading all this. Meanwhile I'll push it all >> out to http://ozlabs.org/~akpm/mmots/ for the crazier testers. >> >> One thing: >> >>> Numbers can be found at https://lkml.org/lkml/2012/9/13/239 >> >> You claim in the above that the fork worload is 'slab intensive". Or >> at least, you seem to - it's a bit fuzzy. >> >> But how slab intensive is it, really? >> >> What is extremely slab intensive is networking. The networking guys >> are very sensitive to slab performance. If this hasn't already been >> done, could you please determine what impact this has upon networking? >> I expect Eric Dumazet, Dave Miller and Tom Herbert could suggest >> testing approaches. >> > > I can test it, but unfortunately I am unlikely to get to prepare a good > environment before Barcelona. > > I know, however, that Greg Thelen was testing netperf in his setup. > Greg, do you have any publishable numbers you could share? Below is my humble opinion. I am worrying about data cache footprint which is possibly caused by this patchset, especially slab implementation. If there are several memcg cgroups, each cgroup has it's own kmem_caches. When each group do slab-intensive job hard, data cache may be overflowed easily, and cache miss rate will be high, therefore this would decrease system performance highly. Is there any result about this? Thanks. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v6 00/29] kmem controller for memcg.
On Fri, Nov 2, 2012 at 2:04 AM, Andrew Morton wrote: > One thing: > >> Numbers can be found at https://lkml.org/lkml/2012/9/13/239 > > You claim in the above that the fork worload is 'slab intensive". Or > at least, you seem to - it's a bit fuzzy. > > But how slab intensive is it, really? > > What is extremely slab intensive is networking. The networking guys > are very sensitive to slab performance. If this hasn't already been > done, could you please determine what impact this has upon networking? > I expect Eric Dumazet, Dave Miller and Tom Herbert could suggest > testing approaches. IIRC, networking guys have reduced their dependency on slab performance recently. Few simple benchmarks to run are hackbench, netperf, and Christoph's famous microbenchmarks. The sad reality is that you usually have to wait for few release cycles before people notice that you've destroyed performance of their favourite workload. :-/ -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v6 00/29] kmem controller for memcg.
On 11/02/2012 04:04 AM, Andrew Morton wrote: > On Thu, 1 Nov 2012 16:07:16 +0400 > Glauber Costa wrote: > >> Hi, >> >> This work introduces the kernel memory controller for memcg. Unlike previous >> submissions, this includes the whole controller, comprised of slab and stack >> memory. > > I'm in the middle of (re)reading all this. Meanwhile I'll push it all > out to http://ozlabs.org/~akpm/mmots/ for the crazier testers. > > One thing: > >> Numbers can be found at https://lkml.org/lkml/2012/9/13/239 > > You claim in the above that the fork worload is 'slab intensive". Or > at least, you seem to - it's a bit fuzzy. > > But how slab intensive is it, really? > > What is extremely slab intensive is networking. The networking guys > are very sensitive to slab performance. If this hasn't already been > done, could you please determine what impact this has upon networking? > I expect Eric Dumazet, Dave Miller and Tom Herbert could suggest > testing approaches. > I can test it, but unfortunately I am unlikely to get to prepare a good environment before Barcelona. I know, however, that Greg Thelen was testing netperf in his setup. Greg, do you have any publishable numbers you could share? -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v6 00/29] kmem controller for memcg.
On Thu, 1 Nov 2012 16:07:16 +0400 Glauber Costa wrote: > Hi, > > This work introduces the kernel memory controller for memcg. Unlike previous > submissions, this includes the whole controller, comprised of slab and stack > memory. I'm in the middle of (re)reading all this. Meanwhile I'll push it all out to http://ozlabs.org/~akpm/mmots/ for the crazier testers. One thing: > Numbers can be found at https://lkml.org/lkml/2012/9/13/239 You claim in the above that the fork worload is 'slab intensive". Or at least, you seem to - it's a bit fuzzy. But how slab intensive is it, really? What is extremely slab intensive is networking. The networking guys are very sensitive to slab performance. If this hasn't already been done, could you please determine what impact this has upon networking? I expect Eric Dumazet, Dave Miller and Tom Herbert could suggest testing approaches. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
