Re: [patch v2 3/6] memcg: rework mem_cgroup_iter to use cgroup iterators

2012-12-14 Thread Ying Han
On Fri, Dec 14, 2012 at 4:07 AM, Michal Hocko wrote: > On Thu 13-12-12 17:14:13, Ying Han wrote: > [...] >> I haven't tried this patch set yet. Before I am doing that, I am >> curious whether changing the target reclaim to be consistent with >> global reclaim something wor

Re: [patch v2 3/6] memcg: rework mem_cgroup_iter to use cgroup iterators

2012-12-14 Thread Ying Han
On Fri, Dec 14, 2012 at 4:07 AM, Michal Hocko mho...@suse.cz wrote: On Thu 13-12-12 17:14:13, Ying Han wrote: [...] I haven't tried this patch set yet. Before I am doing that, I am curious whether changing the target reclaim to be consistent with global reclaim something worthy to consider

Re: [patch v2 3/6] memcg: rework mem_cgroup_iter to use cgroup iterators

2012-12-13 Thread Ying Han
On Wed, Dec 12, 2012 at 11:24 AM, Michal Hocko wrote: > On Wed 12-12-12 10:06:52, Michal Hocko wrote: >> On Tue 11-12-12 14:36:10, Ying Han wrote: > [...] >> > One exception is mem_cgroup_iter_break(), where the loop terminates >> > with *leaked* refcnt and that

Re: [patch v2 3/6] memcg: rework mem_cgroup_iter to use cgroup iterators

2012-12-13 Thread Ying Han
On Wed, Dec 12, 2012 at 10:42 AM, Michal Hocko wrote: > On Wed 12-12-12 19:34:46, Michal Hocko wrote: >> On Wed 12-12-12 10:09:43, Ying Han wrote: >> [...] >> > But If i look at the callers of mem_cgroup_iter(), they all look like >> > the following: >> &g

Re: [patch v2 3/6] memcg: rework mem_cgroup_iter to use cgroup iterators

2012-12-13 Thread Ying Han
On Wed, Dec 12, 2012 at 10:42 AM, Michal Hocko mho...@suse.cz wrote: On Wed 12-12-12 19:34:46, Michal Hocko wrote: On Wed 12-12-12 10:09:43, Ying Han wrote: [...] But If i look at the callers of mem_cgroup_iter(), they all look like the following: memcg = mem_cgroup_iter(root, NULL

Re: [patch v2 3/6] memcg: rework mem_cgroup_iter to use cgroup iterators

2012-12-13 Thread Ying Han
On Wed, Dec 12, 2012 at 11:24 AM, Michal Hocko mho...@suse.cz wrote: On Wed 12-12-12 10:06:52, Michal Hocko wrote: On Tue 11-12-12 14:36:10, Ying Han wrote: [...] One exception is mem_cgroup_iter_break(), where the loop terminates with *leaked* refcnt and that is what the iter_break() needs

Re: [patch v2 3/6] memcg: rework mem_cgroup_iter to use cgroup iterators

2012-12-12 Thread Ying Han
On Wed, Dec 12, 2012 at 1:06 AM, Michal Hocko wrote: > On Tue 11-12-12 14:36:10, Ying Han wrote: >> On Tue, Dec 11, 2012 at 7:54 AM, Michal Hocko wrote: >> > On Sun 09-12-12 11:39:50, Ying Han wrote: >> >> On Mon, Nov 26, 2012 at

Re: [patch v2 3/6] memcg: rework mem_cgroup_iter to use cgroup iterators

2012-12-12 Thread Ying Han
On Wed, Dec 12, 2012 at 12:55 AM, Michal Hocko wrote: > On Tue 11-12-12 14:43:37, Ying Han wrote: >> On Tue, Dec 11, 2012 at 8:15 AM, Michal Hocko wrote: >> > On Tue 11-12-12 16:50:25, Michal Hocko wrote: >> >> On Sun 09-12-12 08:59:54, Ying Han wrote: >> &

Re: [patch v2 3/6] memcg: rework mem_cgroup_iter to use cgroup iterators

2012-12-12 Thread Ying Han
On Wed, Dec 12, 2012 at 12:55 AM, Michal Hocko mho...@suse.cz wrote: On Tue 11-12-12 14:43:37, Ying Han wrote: On Tue, Dec 11, 2012 at 8:15 AM, Michal Hocko mho...@suse.cz wrote: On Tue 11-12-12 16:50:25, Michal Hocko wrote: On Sun 09-12-12 08:59:54, Ying Han wrote: On Mon, Nov 26, 2012

Re: [patch v2 3/6] memcg: rework mem_cgroup_iter to use cgroup iterators

2012-12-12 Thread Ying Han
On Wed, Dec 12, 2012 at 1:06 AM, Michal Hocko mho...@suse.cz wrote: On Tue 11-12-12 14:36:10, Ying Han wrote: On Tue, Dec 11, 2012 at 7:54 AM, Michal Hocko mho...@suse.cz wrote: On Sun 09-12-12 11:39:50, Ying Han wrote: On Mon, Nov 26, 2012 at 10:47 AM, Michal Hocko mho...@suse.cz wrote

Re: [patch v2 4/6] memcg: simplify mem_cgroup_iter

2012-12-11 Thread Ying Han
On Tue, Dec 11, 2012 at 8:01 AM, Michal Hocko wrote: > On Mon 10-12-12 20:35:20, Ying Han wrote: >> On Mon, Nov 26, 2012 at 10:47 AM, Michal Hocko wrote: >> > Current implementation of mem_cgroup_iter has to consider both css and >> > memcg to find out whether no grou

Re: [patch v2 3/6] memcg: rework mem_cgroup_iter to use cgroup iterators

2012-12-11 Thread Ying Han
On Tue, Dec 11, 2012 at 8:15 AM, Michal Hocko wrote: > On Tue 11-12-12 16:50:25, Michal Hocko wrote: >> On Sun 09-12-12 08:59:54, Ying Han wrote: >> > On Mon, Nov 26, 2012 at 10:47 AM, Michal Hocko wrote: >> [...] >> > > + /* >> > >

Re: [patch v2 3/6] memcg: rework mem_cgroup_iter to use cgroup iterators

2012-12-11 Thread Ying Han
On Tue, Dec 11, 2012 at 7:54 AM, Michal Hocko wrote: > On Sun 09-12-12 11:39:50, Ying Han wrote: >> On Mon, Nov 26, 2012 at 10:47 AM, Michal Hocko wrote: > [...] >> > if (reclaim) { >> > - iter->position = id; >> >

Re: [patch v2 3/6] memcg: rework mem_cgroup_iter to use cgroup iterators

2012-12-11 Thread Ying Han
On Tue, Dec 11, 2012 at 7:50 AM, Michal Hocko wrote: > On Sun 09-12-12 08:59:54, Ying Han wrote: >> On Mon, Nov 26, 2012 at 10:47 AM, Michal Hocko wrote: > [...] >> > + /* >> > +* Even if we found a group we have

Re: [patch v2 3/6] memcg: rework mem_cgroup_iter to use cgroup iterators

2012-12-11 Thread Ying Han
On Tue, Dec 11, 2012 at 7:50 AM, Michal Hocko mho...@suse.cz wrote: On Sun 09-12-12 08:59:54, Ying Han wrote: On Mon, Nov 26, 2012 at 10:47 AM, Michal Hocko mho...@suse.cz wrote: [...] + /* +* Even if we found a group we have to make sure it is alive

Re: [patch v2 3/6] memcg: rework mem_cgroup_iter to use cgroup iterators

2012-12-11 Thread Ying Han
On Tue, Dec 11, 2012 at 7:54 AM, Michal Hocko mho...@suse.cz wrote: On Sun 09-12-12 11:39:50, Ying Han wrote: On Mon, Nov 26, 2012 at 10:47 AM, Michal Hocko mho...@suse.cz wrote: [...] if (reclaim) { - iter-position = id

Re: [patch v2 3/6] memcg: rework mem_cgroup_iter to use cgroup iterators

2012-12-11 Thread Ying Han
On Tue, Dec 11, 2012 at 8:15 AM, Michal Hocko mho...@suse.cz wrote: On Tue 11-12-12 16:50:25, Michal Hocko wrote: On Sun 09-12-12 08:59:54, Ying Han wrote: On Mon, Nov 26, 2012 at 10:47 AM, Michal Hocko mho...@suse.cz wrote: [...] + /* +* Even if we found

Re: [patch v2 4/6] memcg: simplify mem_cgroup_iter

2012-12-11 Thread Ying Han
On Tue, Dec 11, 2012 at 8:01 AM, Michal Hocko mho...@suse.cz wrote: On Mon 10-12-12 20:35:20, Ying Han wrote: On Mon, Nov 26, 2012 at 10:47 AM, Michal Hocko mho...@suse.cz wrote: Current implementation of mem_cgroup_iter has to consider both css and memcg to find out whether no group has

Re: [patch v2 4/6] memcg: simplify mem_cgroup_iter

2012-12-10 Thread Ying Han
On Mon, Nov 26, 2012 at 10:47 AM, Michal Hocko wrote: > Current implementation of mem_cgroup_iter has to consider both css and > memcg to find out whether no group has been found (css==NULL - aka the > loop is completed) and that no memcg is associated with the found node > (!memcg - aka

Re: [patch v2 4/6] memcg: simplify mem_cgroup_iter

2012-12-10 Thread Ying Han
On Mon, Nov 26, 2012 at 10:47 AM, Michal Hocko mho...@suse.cz wrote: Current implementation of mem_cgroup_iter has to consider both css and memcg to find out whether no group has been found (css==NULL - aka the loop is completed) and that no memcg is associated with the found node (!memcg -

Re: [patch v2 3/6] memcg: rework mem_cgroup_iter to use cgroup iterators

2012-12-09 Thread Ying Han
On Mon, Nov 26, 2012 at 10:47 AM, Michal Hocko wrote: > mem_cgroup_iter curently relies on css->id when walking down a group > hierarchy tree. This is really awkward because the tree walk depends on > the groups creation ordering. The only guarantee is that a parent node > is visited before its

Re: [patch v2 4/6] memcg: simplify mem_cgroup_iter

2012-12-09 Thread Ying Han
On Mon, Nov 26, 2012 at 10:47 AM, Michal Hocko wrote: > Current implementation of mem_cgroup_iter has to consider both css and > memcg to find out whether no group has been found (css==NULL - aka the > loop is completed) and that no memcg is associated with the found node > (!memcg - aka

Re: [patch v2 3/6] memcg: rework mem_cgroup_iter to use cgroup iterators

2012-12-09 Thread Ying Han
On Mon, Nov 26, 2012 at 10:47 AM, Michal Hocko wrote: > mem_cgroup_iter curently relies on css->id when walking down a group > hierarchy tree. This is really awkward because the tree walk depends on > the groups creation ordering. The only guarantee is that a parent node > is visited before its

Re: [patch v2 3/6] memcg: rework mem_cgroup_iter to use cgroup iterators

2012-12-09 Thread Ying Han
On Mon, Nov 26, 2012 at 10:47 AM, Michal Hocko mho...@suse.cz wrote: mem_cgroup_iter curently relies on css-id when walking down a group hierarchy tree. This is really awkward because the tree walk depends on the groups creation ordering. The only guarantee is that a parent node is visited

Re: [patch v2 4/6] memcg: simplify mem_cgroup_iter

2012-12-09 Thread Ying Han
On Mon, Nov 26, 2012 at 10:47 AM, Michal Hocko mho...@suse.cz wrote: Current implementation of mem_cgroup_iter has to consider both css and memcg to find out whether no group has been found (css==NULL - aka the loop is completed) and that no memcg is associated with the found node (!memcg -

Re: [patch v2 3/6] memcg: rework mem_cgroup_iter to use cgroup iterators

2012-12-09 Thread Ying Han
On Mon, Nov 26, 2012 at 10:47 AM, Michal Hocko mho...@suse.cz wrote: mem_cgroup_iter curently relies on css-id when walking down a group hierarchy tree. This is really awkward because the tree walk depends on the groups creation ordering. The only guarantee is that a parent node is visited

Re: [patch v2 3/6] memcg: rework mem_cgroup_iter to use cgroup iterators

2012-12-07 Thread Ying Han
On Fri, Dec 7, 2012 at 9:27 AM, Michal Hocko wrote: > On Fri 07-12-12 09:12:25, Ying Han wrote: >> On Fri, Dec 7, 2012 at 12:58 AM, Michal Hocko wrote: >> > On Thu 06-12-12 19:43:52, Ying Han wrote: >> > [...] >> >> Forgot to mention, I was test

Re: [patch v2 3/6] memcg: rework mem_cgroup_iter to use cgroup iterators

2012-12-07 Thread Ying Han
On Fri, Dec 7, 2012 at 12:58 AM, Michal Hocko wrote: > On Thu 06-12-12 19:43:52, Ying Han wrote: > [...] >> Forgot to mention, I was testing 3.7-rc6 with the two cgroup changes : > > Could you give a try to -mm tree as well. There are some changes for > memcgs remov

Re: [patch v2 3/6] memcg: rework mem_cgroup_iter to use cgroup iterators

2012-12-07 Thread Ying Han
On Fri, Dec 7, 2012 at 12:58 AM, Michal Hocko mho...@suse.cz wrote: On Thu 06-12-12 19:43:52, Ying Han wrote: [...] Forgot to mention, I was testing 3.7-rc6 with the two cgroup changes : Could you give a try to -mm tree as well. There are some changes for memcgs removal in that tree which

Re: [patch v2 3/6] memcg: rework mem_cgroup_iter to use cgroup iterators

2012-12-07 Thread Ying Han
On Fri, Dec 7, 2012 at 9:27 AM, Michal Hocko mho...@suse.cz wrote: On Fri 07-12-12 09:12:25, Ying Han wrote: On Fri, Dec 7, 2012 at 12:58 AM, Michal Hocko mho...@suse.cz wrote: On Thu 06-12-12 19:43:52, Ying Han wrote: [...] Forgot to mention, I was testing 3.7-rc6 with the two cgroup

Re: [patch v2 3/6] memcg: rework mem_cgroup_iter to use cgroup iterators

2012-12-06 Thread Ying Han
On Thu, Dec 6, 2012 at 7:39 PM, Ying Han wrote: > On Mon, Nov 26, 2012 at 10:47 AM, Michal Hocko wrote: >> mem_cgroup_iter curently relies on css->id when walking down a group >> hierarchy tree. This is really awkward because the tree walk depends on >> the groups cre

Re: [patch v2 3/6] memcg: rework mem_cgroup_iter to use cgroup iterators

2012-12-06 Thread Ying Han
On Mon, Nov 26, 2012 at 10:47 AM, Michal Hocko wrote: > mem_cgroup_iter curently relies on css->id when walking down a group > hierarchy tree. This is really awkward because the tree walk depends on > the groups creation ordering. The only guarantee is that a parent node > is visited before its

Re: [patch v2 3/6] memcg: rework mem_cgroup_iter to use cgroup iterators

2012-12-06 Thread Ying Han
On Mon, Nov 26, 2012 at 10:47 AM, Michal Hocko mho...@suse.cz wrote: mem_cgroup_iter curently relies on css-id when walking down a group hierarchy tree. This is really awkward because the tree walk depends on the groups creation ordering. The only guarantee is that a parent node is visited

Re: [patch v2 3/6] memcg: rework mem_cgroup_iter to use cgroup iterators

2012-12-06 Thread Ying Han
On Thu, Dec 6, 2012 at 7:39 PM, Ying Han ying...@google.com wrote: On Mon, Nov 26, 2012 at 10:47 AM, Michal Hocko mho...@suse.cz wrote: mem_cgroup_iter curently relies on css-id when walking down a group hierarchy tree. This is really awkward because the tree walk depends on the groups

Re: [PATCH v2 00/11] Request for Inclusion: kmem controller for memcg.

2012-08-17 Thread Ying Han
On Thu, Aug 9, 2012 at 6:01 AM, Glauber Costa wrote: > Hi, > > This is the first part of the kernel memory controller for memcg. It has been > discussed many times, and I consider this stable enough to be on tree. A > follow > up to this series are the patches to also track slab memory. They are

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-17 Thread Ying Han
On Thu, Aug 16, 2012 at 8:25 AM, Michal Hocko wrote: > On Wed 15-08-12 12:50:55, Ying Han wrote: >> On Tue, Aug 14, 2012 at 9:21 AM, Michal Hocko wrote: >> > On Thu 09-08-12 17:01:12, Glauber Costa wrote: >> >> This patch adds the basic infrastructure for the accou

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-17 Thread Ying Han
On Thu, Aug 16, 2012 at 8:25 AM, Michal Hocko mho...@suse.cz wrote: On Wed 15-08-12 12:50:55, Ying Han wrote: On Tue, Aug 14, 2012 at 9:21 AM, Michal Hocko mho...@suse.cz wrote: On Thu 09-08-12 17:01:12, Glauber Costa wrote: This patch adds the basic infrastructure for the accounting

Re: [PATCH v2 00/11] Request for Inclusion: kmem controller for memcg.

2012-08-17 Thread Ying Han
On Thu, Aug 9, 2012 at 6:01 AM, Glauber Costa glom...@parallels.com wrote: Hi, This is the first part of the kernel memory controller for memcg. It has been discussed many times, and I consider this stable enough to be on tree. A follow up to this series are the patches to also track slab

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Ying Han
On Tue, Aug 14, 2012 at 9:21 AM, Michal Hocko wrote: > On Thu 09-08-12 17:01:12, Glauber Costa wrote: >> This patch adds the basic infrastructure for the accounting of the slab >> caches. To control that, the following files are created: >> >> * memory.kmem.usage_in_bytes >> *

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Ying Han
does, in general. We assume that if he wants >> >>> that memory and we can serve it, we should. Also, not all kernel memory >> >>> is unreclaimable. We can shrink the slabs, for instance. Ying Han >> >>> claims she has patches for that already... >> >

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Ying Han
can serve it, we should. Also, not all kernel memory >>>> is unreclaimable. We can shrink the slabs, for instance. Ying Han >>>> claims she has patches for that already... >>> >>> Are those patches somewhere around? >> >> You can already shrink th

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Ying Han
; point of view)? >> >> That is not what the kernel does, in general. We assume that if he wants >> that memory and we can serve it, we should. Also, not all kernel memory >> is unreclaimable. We can shrink the slabs, for instance. Ying Han >> claims she has patches

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Ying Han
of the kernel allocation (which is unreclaimable from hard limit reclaim point of view)? That is not what the kernel does, in general. We assume that if he wants that memory and we can serve it, we should. Also, not all kernel memory is unreclaimable. We can shrink the slabs, for instance. Ying Han

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Ying Han
, not all kernel memory is unreclaimable. We can shrink the slabs, for instance. Ying Han claims she has patches for that already... Are those patches somewhere around? You can already shrink the reclaimable slabs (dentries / inodes) via calls to the subsystem specific shrinkers. Did Ying Han do

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Ying Han
that memory and we can serve it, we should. Also, not all kernel memory is unreclaimable. We can shrink the slabs, for instance. Ying Han claims she has patches for that already... Are those patches somewhere around? You can already shrink the reclaimable slabs (dentries / inodes) via calls

Re: [PATCH v2 04/11] kmem accounting basic infrastructure

2012-08-15 Thread Ying Han
On Tue, Aug 14, 2012 at 9:21 AM, Michal Hocko mho...@suse.cz wrote: On Thu 09-08-12 17:01:12, Glauber Costa wrote: This patch adds the basic infrastructure for the accounting of the slab caches. To control that, the following files are created: * memory.kmem.usage_in_bytes *