Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-02-13 Thread Michal Hocko
On Wed 13-02-13 11:34:59, Michal Hocko wrote: > On Tue 12-02-13 12:37:41, Johannes Weiner wrote: > > On Tue, Feb 12, 2013 at 06:12:16PM +0100, Michal Hocko wrote: > [...] > > > diff --git a/mm/memcontrol.c b/mm/memcontrol.c > > > index 727ec39..31bb9b0 100644 > > > --- a/mm/memcontrol.c > > > +++ b

Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-02-13 Thread Michal Hocko
On Wed 13-02-13 12:11:59, Glauber Costa wrote: > On 02/12/2013 09:37 PM, Johannes Weiner wrote: > >> > All reads from root->dead_count are atomic already, so I am not sure > >> > what you mean here. Anyway, I hope I won't make this even more confusing > >> > if I post what I have right now: > > Yes

Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-02-13 Thread Michal Hocko
On Tue 12-02-13 12:37:41, Johannes Weiner wrote: > On Tue, Feb 12, 2013 at 06:12:16PM +0100, Michal Hocko wrote: [...] > > diff --git a/mm/memcontrol.c b/mm/memcontrol.c > > index 727ec39..31bb9b0 100644 > > --- a/mm/memcontrol.c > > +++ b/mm/memcontrol.c > > @@ -144,8 +144,13 @@ struct mem_cgroup_

Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-02-13 Thread Michal Hocko
On Tue 12-02-13 14:53:58, Johannes Weiner wrote: [...] > iteration: > rcu_read_lock() > dead_count = atomic_read(&hierarchy->dead_count) > smp_rmb() > previous = iterator->position > if (iterator->dead_count != dead_count) >/* A cgroup in our hierarchy was killed, pointer might be dangling */ >

Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-02-13 Thread Glauber Costa
On 02/12/2013 09:37 PM, Johannes Weiner wrote: >> > All reads from root->dead_count are atomic already, so I am not sure >> > what you mean here. Anyway, I hope I won't make this even more confusing >> > if I post what I have right now: > Yes, but we are doing two reads. Can't the memcg that we'll

Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-02-12 Thread Johannes Weiner
On Tue, Feb 12, 2013 at 10:31:48AM -0800, Paul E. McKenney wrote: > On Tue, Feb 12, 2013 at 12:25:26PM -0500, Johannes Weiner wrote: > > On Tue, Feb 12, 2013 at 08:10:51AM -0800, Paul E. McKenney wrote: > > > On Tue, Feb 12, 2013 at 04:43:30PM +0100, Michal Hocko wrote: > > > > On Tue 12-02-13 10:1

Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-02-12 Thread Paul E. McKenney
On Tue, Feb 12, 2013 at 12:25:26PM -0500, Johannes Weiner wrote: > On Tue, Feb 12, 2013 at 08:10:51AM -0800, Paul E. McKenney wrote: > > On Tue, Feb 12, 2013 at 04:43:30PM +0100, Michal Hocko wrote: > > > On Tue 12-02-13 10:10:02, Johannes Weiner wrote: > > > > On Tue, Feb 12, 2013 at 10:54:19AM +0

Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-02-12 Thread Michal Hocko
On Tue 12-02-13 08:10:51, Paul E. McKenney wrote: > On Tue, Feb 12, 2013 at 04:43:30PM +0100, Michal Hocko wrote: > > On Tue 12-02-13 10:10:02, Johannes Weiner wrote: > > > On Tue, Feb 12, 2013 at 10:54:19AM +0100, Michal Hocko wrote: > > > > On Mon 11-02-13 17:39:43, Johannes Weiner wrote: > > > >

Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-02-12 Thread Paul E. McKenney
On Tue, Feb 12, 2013 at 04:43:30PM +0100, Michal Hocko wrote: > On Tue 12-02-13 10:10:02, Johannes Weiner wrote: > > On Tue, Feb 12, 2013 at 10:54:19AM +0100, Michal Hocko wrote: > > > On Mon 11-02-13 17:39:43, Johannes Weiner wrote: > > > > On Mon, Feb 11, 2013 at 10:27:56PM +0100, Michal Hocko wr

Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-02-12 Thread Johannes Weiner
On Tue, Feb 12, 2013 at 06:12:16PM +0100, Michal Hocko wrote: > On Tue 12-02-13 11:41:03, Johannes Weiner wrote: > > > > > > Michal Hocko wrote: > > > > >On Tue 12-02-13 17:13:32, Michal Hocko wrote: > > >> On Tue 12-02-13 16:43:30, Michal Hocko wrote: > > >> [...] > > >> The example was not co

Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-02-12 Thread Johannes Weiner
On Tue, Feb 12, 2013 at 08:10:51AM -0800, Paul E. McKenney wrote: > On Tue, Feb 12, 2013 at 04:43:30PM +0100, Michal Hocko wrote: > > On Tue 12-02-13 10:10:02, Johannes Weiner wrote: > > > On Tue, Feb 12, 2013 at 10:54:19AM +0100, Michal Hocko wrote: > > > > On Mon 11-02-13 17:39:43, Johannes Weine

Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-02-12 Thread Michal Hocko
On Tue 12-02-13 11:41:03, Johannes Weiner wrote: > > > Michal Hocko wrote: > > >On Tue 12-02-13 17:13:32, Michal Hocko wrote: > >> On Tue 12-02-13 16:43:30, Michal Hocko wrote: > >> [...] > >> The example was not complete: > >> > >> > Wait a moment. But what prevents from the following race? >

Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-02-12 Thread Johannes Weiner
Michal Hocko wrote: >On Tue 12-02-13 17:13:32, Michal Hocko wrote: >> On Tue 12-02-13 16:43:30, Michal Hocko wrote: >> [...] >> The example was not complete: >> >> > Wait a moment. But what prevents from the following race? >> > >> > rcu_read_lock() >> >> cgroup_next_descendant_pre >> css_tr

Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-02-12 Thread Michal Hocko
On Tue 12-02-13 17:24:42, Michal Hocko wrote: > On Tue 12-02-13 17:13:32, Michal Hocko wrote: > > On Tue 12-02-13 16:43:30, Michal Hocko wrote: > > [...] > > The example was not complete: > > > > > Wait a moment. But what prevents from the following race? > > > > > > rcu_read_lock() > > > > cgro

Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-02-12 Thread Johannes Weiner
Michal Hocko wrote: >On Tue 12-02-13 10:10:02, Johannes Weiner wrote: >> On Tue, Feb 12, 2013 at 10:54:19AM +0100, Michal Hocko wrote: >> > On Mon 11-02-13 17:39:43, Johannes Weiner wrote: >> > > On Mon, Feb 11, 2013 at 10:27:56PM +0100, Michal Hocko wrote: >> > > > On Mon 11-02-13 14:58:24, Jo

Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-02-12 Thread Michal Hocko
On Tue 12-02-13 17:13:32, Michal Hocko wrote: > On Tue 12-02-13 16:43:30, Michal Hocko wrote: > [...] > The example was not complete: > > > Wait a moment. But what prevents from the following race? > > > > rcu_read_lock() > > cgroup_next_descendant_pre > css_tryget(css); > memcg = mem_cgroup_fro

Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-02-12 Thread Michal Hocko
On Tue 12-02-13 16:43:30, Michal Hocko wrote: [...] The example was not complete: > Wait a moment. But what prevents from the following race? > > rcu_read_lock() cgroup_next_descendant_pre css_tryget(css); memcg = mem_cgroup_from_css(css)atomic_add(CSS_DEACT_BIAS, &css->refcnt)

Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-02-12 Thread Michal Hocko
On Tue 12-02-13 10:10:02, Johannes Weiner wrote: > On Tue, Feb 12, 2013 at 10:54:19AM +0100, Michal Hocko wrote: > > On Mon 11-02-13 17:39:43, Johannes Weiner wrote: > > > On Mon, Feb 11, 2013 at 10:27:56PM +0100, Michal Hocko wrote: > > > > On Mon 11-02-13 14:58:24, Johannes Weiner wrote: > > > >

Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-02-12 Thread Johannes Weiner
On Tue, Feb 12, 2013 at 10:54:19AM +0100, Michal Hocko wrote: > On Mon 11-02-13 17:39:43, Johannes Weiner wrote: > > On Mon, Feb 11, 2013 at 10:27:56PM +0100, Michal Hocko wrote: > > > On Mon 11-02-13 14:58:24, Johannes Weiner wrote: > > > > That way, if the dead count gives the go-ahead, you KNOW

Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-02-12 Thread Michal Hocko
On Mon 11-02-13 17:39:43, Johannes Weiner wrote: > On Mon, Feb 11, 2013 at 10:27:56PM +0100, Michal Hocko wrote: > > On Mon 11-02-13 14:58:24, Johannes Weiner wrote: > > > On Mon, Feb 11, 2013 at 08:29:29PM +0100, Michal Hocko wrote: > > > > On Mon 11-02-13 12:56:19, Johannes Weiner wrote: > > > >

Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-02-11 Thread Johannes Weiner
On Mon, Feb 11, 2013 at 10:27:56PM +0100, Michal Hocko wrote: > On Mon 11-02-13 14:58:24, Johannes Weiner wrote: > > On Mon, Feb 11, 2013 at 08:29:29PM +0100, Michal Hocko wrote: > > > On Mon 11-02-13 12:56:19, Johannes Weiner wrote: > > > > On Mon, Feb 11, 2013 at 04:16:49PM +0100, Michal Hocko wr

Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-02-11 Thread Michal Hocko
On Mon 11-02-13 22:27:56, Michal Hocko wrote: [...] > I will get back to this tomorrow. Maybe not a great idea as it is getting late here and brain turns into cabbage but there we go: --- >From f927358fe620837081d7a7ec6bf27af378deb35d Mon Sep 17 00:00:00 2001 From: Michal Hocko Date: Mon, 11 Feb

Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-02-11 Thread Michal Hocko
On Mon 11-02-13 14:58:24, Johannes Weiner wrote: > On Mon, Feb 11, 2013 at 08:29:29PM +0100, Michal Hocko wrote: > > On Mon 11-02-13 12:56:19, Johannes Weiner wrote: > > > On Mon, Feb 11, 2013 at 04:16:49PM +0100, Michal Hocko wrote: > > > > Maybe we could keep the counter per memcg but that would

Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-02-11 Thread Johannes Weiner
On Mon, Feb 11, 2013 at 08:29:29PM +0100, Michal Hocko wrote: > On Mon 11-02-13 12:56:19, Johannes Weiner wrote: > > On Mon, Feb 11, 2013 at 04:16:49PM +0100, Michal Hocko wrote: > > > Maybe we could keep the counter per memcg but that would mean that we > > > would need to go up the hierarchy as w

Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-02-11 Thread Michal Hocko
On Mon 11-02-13 12:56:19, Johannes Weiner wrote: > On Mon, Feb 11, 2013 at 04:16:49PM +0100, Michal Hocko wrote: > > On Fri 08-02-13 14:33:18, Johannes Weiner wrote: > > [...] > > > for each in hierarchy: > > > for each node: > > > for each zone: > > > for each reclaim priority: > > >

Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-02-11 Thread Johannes Weiner
On Mon, Feb 11, 2013 at 04:16:49PM +0100, Michal Hocko wrote: > On Fri 08-02-13 14:33:18, Johannes Weiner wrote: > [...] > > for each in hierarchy: > > for each node: > > for each zone: > > for each reclaim priority: > > > > every time a cgroup is destroyed. I don't think such a hamme

Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-02-11 Thread Michal Hocko
On Fri 08-02-13 14:33:18, Johannes Weiner wrote: [...] > for each in hierarchy: > for each node: > for each zone: > for each reclaim priority: > > every time a cgroup is destroyed. I don't think such a hammer is > justified in general, let alone for consolidating code a little. > > C

Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-02-08 Thread Johannes Weiner
On Thu, Jan 03, 2013 at 06:54:18PM +0100, Michal Hocko wrote: > Now that per-node-zone-priority iterator caches memory cgroups rather > than their css ids we have to be careful and remove them from the > iterator when they are on the way out otherwise they might hang for > unbounded amount of time

Re: [PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-01-06 Thread Kamezawa Hiroyuki
(2013/01/04 2:54), Michal Hocko wrote: > Now that per-node-zone-priority iterator caches memory cgroups rather > than their css ids we have to be careful and remove them from the > iterator when they are on the way out otherwise they might hang for > unbounded amount of time (until the global/targe

[PATCH v3 4/7] memcg: remove memcg from the reclaim iterators

2013-01-03 Thread Michal Hocko
Now that per-node-zone-priority iterator caches memory cgroups rather than their css ids we have to be careful and remove them from the iterator when they are on the way out otherwise they might hang for unbounded amount of time (until the global/targeted reclaim triggers the zone under priority to