Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-16 Thread Ying Han
On Thu, Aug 16, 2012 at 8:25 AM, Michal Hocko wrote: > On Wed 15-08-12 12:50:55, Ying Han wrote: >> On Tue, Aug 14, 2012 at 9:21 AM, Michal Hocko wrote: >> > On Thu 09-08-12 17:01:12, Glauber Costa wrote: >> >> This patch adds the basic infrastructure for the accounting of the slab >> >> caches.

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-16 Thread Kamezawa Hiroyuki
(2012/08/13 17:36), Glauber Costa wrote: > On 08/10/2012 09:02 PM, Kamezawa Hiroyuki wrote: >> (2012/08/09 22:01), Glauber Costa wrote: >>> This patch adds the basic infrastructure for the accounting of the slab >>> caches. To control that, the following files are created: >>> >>>* memory.kmem.

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-16 Thread Michal Hocko
On Wed 15-08-12 12:50:55, Ying Han wrote: > On Tue, Aug 14, 2012 at 9:21 AM, Michal Hocko wrote: > > On Thu 09-08-12 17:01:12, Glauber Costa wrote: > >> This patch adds the basic infrastructure for the accounting of the slab > >> caches. To control that, the following files are created: > >> > >>

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Ying Han
On Tue, Aug 14, 2012 at 9:21 AM, Michal Hocko wrote: > On Thu 09-08-12 17:01:12, Glauber Costa wrote: >> This patch adds the basic infrastructure for the accounting of the slab >> caches. To control that, the following files are created: >> >> * memory.kmem.usage_in_bytes >> * memory.kmem.limit_

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Glauber Costa
On 08/15/2012 10:25 PM, Christoph Lameter wrote: > On Wed, 15 Aug 2012, Ying Han wrote: > >>> How can you figure out which objects belong to which memcg? The ownerships >>> of dentries and inodes is a dubious concept already. >> >> I figured it out based on the kernel slab accounting. >> obj->page

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Christoph Lameter
On Wed, 15 Aug 2012, Ying Han wrote: > > How can you figure out which objects belong to which memcg? The ownerships > > of dentries and inodes is a dubious concept already. > > I figured it out based on the kernel slab accounting. > obj->page->kmem_cache->memcg Well that is only the memcg which a

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Ying Han
On Wed, Aug 15, 2012 at 8:34 AM, Christoph Lameter wrote: > On Wed, 15 Aug 2012, Glauber Costa wrote: > >> On 08/15/2012 06:47 PM, Christoph Lameter wrote: >> > On Wed, 15 Aug 2012, Michal Hocko wrote: >> > >> >>> That is not what the kernel does, in general. We assume that if he wants >> >>> that

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Ying Han
On Wed, Aug 15, 2012 at 8:11 AM, Glauber Costa wrote: > On 08/15/2012 06:47 PM, Christoph Lameter wrote: >> On Wed, 15 Aug 2012, Michal Hocko wrote: >> That is not what the kernel does, in general. We assume that if he wants that memory and we can serve it, we should. Also, not all kerne

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Glauber Costa
On 08/15/2012 10:01 PM, Ying Han wrote: > On Wed, Aug 15, 2012 at 5:39 AM, Michal Hocko wrote: >> On Wed 15-08-12 13:33:55, Glauber Costa wrote: >> [...] This can be quite confusing. I am still not sure whether we should mix the two things together. If somebody wants to limit the k

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Ying Han
On Wed, Aug 15, 2012 at 5:39 AM, Michal Hocko wrote: > On Wed 15-08-12 13:33:55, Glauber Costa wrote: > [...] >> > This can >> > be quite confusing. I am still not sure whether we should mix the two >> > things together. If somebody wants to limit the kernel memory he has to >> > touch the other

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Christoph Lameter
On Wed, 15 Aug 2012, Glauber Costa wrote: > Remember we copy over the metadata and create copies of the caches > per-memcg. Therefore, a dentry belongs to a memcg if it was allocated > from the slab pertaining to that memcg. The dentry could be used by other processes in the system though. F.e. d

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Christoph Lameter
On Wed, 15 Aug 2012, Greg Thelen wrote: > > You can already shrink the reclaimable slabs (dentries / inodes) via > > calls to the subsystem specific shrinkers. Did Ying Han do anything to > > go beyond that? > > cc: Ying > > The Google shrinker patches enhance prune_dcache_sb() to limit dentry > p

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Christoph Lameter
On Wed, 15 Aug 2012, Glauber Costa wrote: > On 08/15/2012 06:47 PM, Christoph Lameter wrote: > > On Wed, 15 Aug 2012, Michal Hocko wrote: > > > >>> That is not what the kernel does, in general. We assume that if he wants > >>> that memory and we can serve it, we should. Also, not all kernel memory

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Glauber Costa
On 08/15/2012 07:34 PM, Christoph Lameter wrote: > On Wed, 15 Aug 2012, Glauber Costa wrote: > >> On 08/15/2012 06:47 PM, Christoph Lameter wrote: >>> On Wed, 15 Aug 2012, Michal Hocko wrote: >>> > That is not what the kernel does, in general. We assume that if he wants > that memory and w

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Greg Thelen
On Wed, Aug 15 2012, Christoph Lameter wrote: > On Wed, 15 Aug 2012, Michal Hocko wrote: > >> > That is not what the kernel does, in general. We assume that if he wants >> > that memory and we can serve it, we should. Also, not all kernel memory >> > is unreclaimable. We can shrink the slabs, for

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Glauber Costa
On 08/15/2012 06:47 PM, Christoph Lameter wrote: > On Wed, 15 Aug 2012, Michal Hocko wrote: > >>> That is not what the kernel does, in general. We assume that if he wants >>> that memory and we can serve it, we should. Also, not all kernel memory >>> is unreclaimable. We can shrink the slabs, for

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Christoph Lameter
On Wed, 15 Aug 2012, Michal Hocko wrote: > > That is not what the kernel does, in general. We assume that if he wants > > that memory and we can serve it, we should. Also, not all kernel memory > > is unreclaimable. We can shrink the slabs, for instance. Ying Han > > claims she has patches for tha

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Glauber Costa
> OK, I missed an important point that kmem_accounted is not exported to > the userspace (I thought it would be done later in the series) which > is not the case so actually nobody get's confused by the inconsistency > because it is about RESOURCE_MAX which they see in both cases. > Sorry about th

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Michal Hocko
On Wed 15-08-12 17:31:24, Glauber Costa wrote: > On 08/15/2012 05:26 PM, Michal Hocko wrote: > > On Wed 15-08-12 17:04:31, Glauber Costa wrote: > >> On 08/15/2012 05:02 PM, Michal Hocko wrote: > >>> On Wed 15-08-12 16:53:40, Glauber Costa wrote: > >>> [...] > >>> This doesn't check for the hier

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Glauber Costa
On 08/15/2012 05:26 PM, Michal Hocko wrote: > On Wed 15-08-12 17:04:31, Glauber Costa wrote: >> On 08/15/2012 05:02 PM, Michal Hocko wrote: >>> On Wed 15-08-12 16:53:40, Glauber Costa wrote: >>> [...] >>> This doesn't check for the hierachy so kmem_accounted might not be in >>> sync with i

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread James Bottomley
On Wed, 2012-08-15 at 14:55 +0200, Michal Hocko wrote: > On Wed 15-08-12 12:12:23, James Bottomley wrote: > > On Wed, 2012-08-15 at 13:33 +0400, Glauber Costa wrote: > > > > This can > > > > be quite confusing. I am still not sure whether we should mix the two > > > > things together. If somebody

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Michal Hocko
On Wed 15-08-12 17:04:31, Glauber Costa wrote: > On 08/15/2012 05:02 PM, Michal Hocko wrote: > > On Wed 15-08-12 16:53:40, Glauber Costa wrote: > > [...] > > This doesn't check for the hierachy so kmem_accounted might not be in > > sync with it's parents. mem_cgroup_create (below) needs to

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Glauber Costa
On 08/15/2012 05:02 PM, Michal Hocko wrote: > On Wed 15-08-12 16:53:40, Glauber Costa wrote: > [...] > This doesn't check for the hierachy so kmem_accounted might not be in > sync with it's parents. mem_cgroup_create (below) needs to copy > kmem_accounted down from the parent and the a

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Michal Hocko
On Wed 15-08-12 16:53:40, Glauber Costa wrote: [...] > >>> This doesn't check for the hierachy so kmem_accounted might not be in > >>> sync with it's parents. mem_cgroup_create (below) needs to copy > >>> kmem_accounted down from the parent and the above needs to check if this > >>> is a similar d

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Michal Hocko
On Wed 15-08-12 12:12:23, James Bottomley wrote: > On Wed, 2012-08-15 at 13:33 +0400, Glauber Costa wrote: > > > This can > > > be quite confusing. I am still not sure whether we should mix the two > > > things together. If somebody wants to limit the kernel memory he has to > > > touch the other

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Glauber Costa
On 08/15/2012 04:39 PM, Michal Hocko wrote: > On Wed 15-08-12 13:33:55, Glauber Costa wrote: > [...] >>> This can >>> be quite confusing. I am still not sure whether we should mix the two >>> things together. If somebody wants to limit the kernel memory he has to >>> touch the other limit anyway.

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Michal Hocko
On Wed 15-08-12 13:33:55, Glauber Costa wrote: [...] > > This can > > be quite confusing. I am still not sure whether we should mix the two > > things together. If somebody wants to limit the kernel memory he has to > > touch the other limit anyway. Do you have a strong reason to mix the > > user

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread James Bottomley
On Wed, 2012-08-15 at 13:33 +0400, Glauber Costa wrote: > > This can > > be quite confusing. I am still not sure whether we should mix the two > > things together. If somebody wants to limit the kernel memory he has to > > touch the other limit anyway. Do you have a strong reason to mix the > > u

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Glauber Costa
>> We always account to both user and kernel resource_counters. This >> effectively means that an independent kernel limit is in place when the >> limit is set to a lower value than the user memory. A equal or higher >> value means that the user limit will always hit first, meaning that kmem >> is

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-14 Thread Michal Hocko
On Thu 09-08-12 17:01:12, Glauber Costa wrote: > This patch adds the basic infrastructure for the accounting of the slab > caches. To control that, the following files are created: > > * memory.kmem.usage_in_bytes > * memory.kmem.limit_in_bytes > * memory.kmem.failcnt > * memory.kmem.max_usage

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-13 Thread Glauber Costa
On 08/10/2012 09:02 PM, Kamezawa Hiroyuki wrote: > (2012/08/09 22:01), Glauber Costa wrote: >> This patch adds the basic infrastructure for the accounting of the slab >> caches. To control that, the following files are created: >> >> * memory.kmem.usage_in_bytes >> * memory.kmem.limit_in_bytes

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-10 Thread Kamezawa Hiroyuki
(2012/08/09 22:01), Glauber Costa wrote: > This patch adds the basic infrastructure for the accounting of the slab > caches. To control that, the following files are created: > > * memory.kmem.usage_in_bytes > * memory.kmem.limit_in_bytes > * memory.kmem.failcnt > * memory.kmem.max_usage_i

[PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-09 Thread Glauber Costa
This patch adds the basic infrastructure for the accounting of the slab caches. To control that, the following files are created: * memory.kmem.usage_in_bytes * memory.kmem.limit_in_bytes * memory.kmem.failcnt * memory.kmem.max_usage_in_bytes They have the same meaning of their user memory co