Re: [PATCH RESEND -mm 02/12] memcg: fix race in memcg cache destruction path

2014-03-18 Thread Vladimir Davydov
On 03/17/2014 08:42 PM, Michal Hocko wrote: > On Thu 13-03-14 19:06:40, Vladimir Davydov wrote: >> We schedule memcg cache shrink+destruction work (memcg_params::destroy) >> from two places: when we turn memcg offline >> (mem_cgroup_destroy_all_caches) and when the last

Re: [PATCH RESEND -mm 01/12] memcg: flush cache creation works before memcg cache destruction

2014-03-18 Thread Vladimir Davydov
On 03/18/2014 12:55 PM, Michal Hocko wrote: > On Tue 18-03-14 12:14:37, Vladimir Davydov wrote: >> On 03/17/2014 08:07 PM, Michal Hocko wrote: >>> On Thu 13-03-14 19:06:39, Vladimir Davydov wrote: >>>> When we get to memcg cache destruction, either from the root cache

Re: [PATCH RESEND -mm 02/12] memcg: fix race in memcg cache destruction path

2014-03-18 Thread Vladimir Davydov
On 03/18/2014 02:01 PM, Michal Hocko wrote: > On Tue 18-03-14 12:19:00, Vladimir Davydov wrote: >> On 03/17/2014 08:42 PM, Michal Hocko wrote: >>> On Thu 13-03-14 19:06:40, Vladimir Davydov wrote: >>>> We schedule memcg cache shrink+destruction work (memcg_params:

Re: [PATCH -mm v15 00/13] kmemcg shrinkers

2014-02-11 Thread Vladimir Davydov
Hi Michal, Johannes, David, Could you please take a look at this if you have time? Without your review, it'll never get committed. Thank you. On 02/05/2014 10:39 PM, Vladimir Davydov wrote: > Hi, > > This is the 15th iteration of Glauber Costa's patch-set implementing slab &

Re: [PATCH 1/2] kobject: don't block for each kobject_uevent

2014-02-11 Thread Vladimir Davydov
On 02/12/2014 03:03 AM, Andrew Morton wrote: > On Sun, 9 Feb 2014 14:56:15 +0400 Vladimir Davydov > wrote: > >> Currently kobject_uevent has somewhat unpredictable semantics. The point >> is, since it may call a usermode helper and wait for it to execute >> (UMH_WAIT

Re: [PATCH -mm v15 00/13] kmemcg shrinkers

2014-02-12 Thread Vladimir Davydov
On 02/12/2014 12:19 AM, Johannes Weiner wrote: > On Tue, Feb 11, 2014 at 07:15:26PM +0400, Vladimir Davydov wrote: >> Hi Michal, Johannes, David, >> >> Could you please take a look at this if you have time? Without your >> review, it'll never get committed. >

Re: [PATCH -mm v15 00/13] kmemcg shrinkers

2014-02-13 Thread Vladimir Davydov
On 02/13/2014 02:01 AM, Johannes Weiner wrote: > On Wed, Feb 12, 2014 at 10:05:43PM +0400, Vladimir Davydov wrote: >> On 02/12/2014 12:19 AM, Johannes Weiner wrote: >>> On Tue, Feb 11, 2014 at 07:15:26PM +0400, Vladimir Davydov wrote: >>>> Hi Michal, Johannes, Dav

Re: Unkillable R-state process stuck in sendfile

2014-02-18 Thread Vladimir Davydov
On 02/18/2014 11:13 PM, valdis.kletni...@vt.edu wrote: > On Mon, 17 Feb 2014 16:51:37 +0400, Vladimir Davydov said: > >> out_fd = eventfd(0, 0); >> if (out_fd < 0) >> err(1, "eventfd"); >> ret = sendfile64(out_fd,

[PATCH -mm v3 7/7] slub: rework sysfs layout for memcg caches

2014-02-19 Thread Vladimir Davydov
tory. This would allow us to keep sysfs layout clean, and avoid such inconsistencies like one described above. This patch does the trick. It creates a "cgroup" kset in each root cache kobject to keep its children caches there. Signed-off-by: Vladimir Davydov --- include/linux/slub_def.h

[PATCH -mm v3 2/7] memcg, slab: cleanup memcg cache creation

2014-02-19 Thread Vladimir Davydov
creation path is protected by the slab_mutex. - Get rid of memcg_create_kmem_cache(). This function serves as a proxy to kmem_cache_create_memcg(). After separating the cache name creation path, it would be reduced to a function call, so let's inline it. Signed-off-by: Vladimir Da

[PATCH -mm v3 6/7] slub: adjust memcg caches when creating cache alias

2014-02-19 Thread Vladimir Davydov
Otherwise, kzalloc() called from a memcg won't clear the whole object. Signed-off-by: Vladimir Davydov --- mm/slub.c | 13 + 1 file changed, 13 insertions(+) diff --git a/mm/slub.c b/mm/slub.c index 2f40e615368f..cac2ff7fd68c 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -3

[PATCH -mm v3 5/7] memcg, slab: do not destroy children caches if parent has aliases

2014-02-19 Thread Vladimir Davydov
ion after the check if the cache has aliases. Plus, it forbids destroying a root cache if it still has children caches, because each children cache keeps a reference to its parent. Signed-off-by: Vladimir Davydov --- include/linux/memcontrol.h |6 +--- mm/memcontrol.c| 13

[PATCH -mm v3 3/7] memcg, slab: separate memcg vs root cache creation paths

2014-02-19 Thread Vladimir Davydov
Memcg-awareness turned kmem_cache_create() into a dirty interweaving of memcg-only and except-for-memcg calls. To clean this up, let's move the code responsible for memcg cache creation to a separate function. Signed-off-by: Vladimir Davydov --- include/linux/memcontrol.h |6 -- in

[PATCH -mm v3 1/7] memcg, slab: never try to merge memcg caches

2014-02-19 Thread Vladimir Davydov
y are prevented from merging preliminary I guess we should avoid to merge their children too. So let's remove the useless code responsible for merging memcg caches. Signed-off-by: Vladimir Davydov --- mm/slab.h| 21 - mm/slab_common.c |8 +--- mm/

[PATCH -mm v3 4/7] memcg, slab: unregister cache from memcg before starting to destroy it

2014-02-19 Thread Vladimir Davydov
ocess beginning, issuing memcg_register_cache() on failure. Signed-off-by: Vladimir Davydov --- mm/memcontrol.c | 12 ++-- mm/slab_common.c |3 ++- 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 0e11c72047d0..b07e08f97460 100644

[PATCH -mm v3 0/7] memcg-vs-slab related fixes, improvements, cleanups

2014-02-19 Thread Vladimir Davydov
descriptions - overall cleanup - rebase onto v3.14-rc3 Changes in v2 (thanks to David Rientjes): - do not remove cgroup name part from memcg cache names - do not export memcg cache id to userspace Thanks, Vladimir Davydov (7): memcg, slab: never try to merge memcg caches memcg, slab: cleanup memcg

Re: [PATCH 1/3] mm: vmscan: shrink_slab: rename max_pass -> freeable

2014-01-21 Thread Vladimir Davydov
On 01/22/2014 02:22 AM, David Rientjes wrote: > On Fri, 17 Jan 2014, Vladimir Davydov wrote: > >> The name `max_pass' is misleading, because this variable actually keeps >> the estimate number of freeable objects, not the maximal number of >> objects we can scan in

Re: slab: clean up kmem_cache_create_memcg() error handling

2014-01-24 Thread Vladimir Davydov
84818f > > Parent: 309381feaee564281c3d9e90fbca8963bb7428ad > > Author: Vladimir Davydov > > AuthorDate: Thu Jan 23 15:52:55 2014 -0800 > > Committer: Linus Torvalds > > CommitDate: Thu Jan 23 16:36:50 2014 -0800 > > > > slab: clean up kmem_cach

[PATCH] slab: fix wrong retval on kmem_cache_create_memcg error path

2014-01-24 Thread Vladimir Davydov
e will skip the check and return 's' that can contain crap. Let's fix it by assuring that on error path there are always two conditions satisfied at the same time, err != 0 and s == NULL, by explicitly zeroing 's' after freeing it on error path. Signed-off-by: Dave Jones

Re: [PATCH] slab: fix wrong retval on kmem_cache_create_memcg error path

2014-01-26 Thread Vladimir Davydov
On 01/26/2014 08:39 AM, David Rientjes wrote: > On Sat, 25 Jan 2014, Vladimir Davydov wrote: > >> diff --git a/mm/slab_common.c b/mm/slab_common.c >> index 8e40321..499b53c 100644 >> --- a/mm/slab_common.c >> +++ b/mm/slab_common.c >

Re: [PATCH -mm v2 2/2] mm: get rid of __GFP_KMEMCG

2014-04-01 Thread Vladimir Davydov
On 04/02/2014 04:48 AM, Greg Thelen wrote: > diff --git a/include/linux/slab.h b/include/linux/slab.h > index 3dd389aa91c7..6d6959292e00 100644 > --- a/include/linux/slab.h > +++ b/include/linux/slab.h > @@ -358,17 +358,6 @@ kmem_cache_alloc_node_trace(struct kmem_cache *s, > #include > #endif >

[PATCH -mm v2.1] mm: get rid of __GFP_KMEMCG

2014-04-01 Thread Vladimir Davydov
only differ from alloc_pages and free_pages in that besides allocating or freeing pages they also charge them to the kmem resource counter of the current memory cgroup. Signed-off-by: Vladimir Davydov Cc: Johannes Weiner Cc: Michal Hocko Cc: Glauber Costa Cc: Christoph Lameter Cc: Pekka E

Re: [PATCH -mm v2.1] mm: get rid of __GFP_KMEMCG

2014-04-03 Thread Vladimir Davydov
On 04/03/2014 01:25 AM, Greg Thelen wrote: > On Tue, Apr 01 2014, Vladimir Davydov wrote: > >> Currently to allocate a page that should be charged to kmemcg (e.g. >> threadinfo), we pass __GFP_KMEMCG flag to the page allocator. The page >> allocated is then to be freed

[PATCH -mm v2.2] mm: get rid of __GFP_KMEMCG

2014-04-03 Thread Vladimir Davydov
only differ from alloc_pages and free_pages in that besides allocating or freeing pages they also charge them to the kmem resource counter of the current memory cgroup. Signed-off-by: Vladimir Davydov Acked-by: Greg Thelen Cc: Johannes Weiner Cc: Michal Hocko Cc: Glauber Costa Cc: Christoph La

[PATCH -mm 2/3] lockdep: mark rwsem_acquire_read as recursive

2014-04-06 Thread Vladimir Davydov
nk rw semaphore is read-recursive, just like rw spin lock. Signed-off-by: Vladimir Davydov Cc: Peter Zijlstra Cc: Ingo Molnar --- include/linux/lockdep.h |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/lockdep.h b/include/linux/lockdep.h index 008388f

[PATCH -mm 0/3] slab: cleanup mem hotplug synchronization

2014-04-06 Thread Vladimir Davydov
rong ordering with a sysfs internal mutex in case of slub, because in contrast to recursive read lock, non-recursive one should always be taken in the same order with a mutex. That's why in patch 2 I mark rw semaphore as read-recursive, just like rw spin lock. Thanks, Vladimir Davydov

[PATCH -mm 3/3] slab: lock_memory_hotplug for kmem_cache_{create,destroy,shrink}

2014-04-06 Thread Vladimir Davydov
As a result we'll get a kmem cache with not all kmem_cache_nodes allocated. To avoid issues like that we should hold the memory hotplug lock during the whole kmem cache creation/destruction/shrink paths, just like we do with cpu hotplug. This patch does the trick. Signed-off-by: Vladimir D

[PATCH -mm 1/3] mem-hotplug: turn mem_hotplug_mutex to rwsem

2014-04-06 Thread Vladimir Davydov
ex to an rw sempahore so that lock_memory_hotplug only takes it for reading while the memory hotplug code locks it for writing. This will allow to invoke code under {un}lock_memory_hotplug concurrently while still guaranteeing protection against memory hotplug. Signed-off-by: Vladimir Da

Re: [PATCH -mm 0/3] slab: cleanup mem hotplug synchronization

2014-04-06 Thread Vladimir Davydov
On 04/06/2014 07:33 PM, Vladimir Davydov wrote: > kmem_cache_{create,destroy,shrink} need to get a stable value of > cpu/node online mask, because they init/destroy/access per-cpu/node > kmem_cache parts, which can be allocated or destroyed on cpu/mem > hotplug. To protect against

Re: [PATCH -mm 2/3] lockdep: mark rwsem_acquire_read as recursive

2014-04-07 Thread Vladimir Davydov
On 04/07/2014 12:13 PM, Peter Zijlstra wrote: > On Sun, Apr 06, 2014 at 07:33:51PM +0400, Vladimir Davydov wrote: >> rw_semaphore implementation allows recursing calls to down_read, but >> lockdep thinks that it doesn't. As a result, it will complain >> false-positively,

[PATCH -mm v2 0/2] slab: cleanup mem hotplug synchronization

2014-04-07 Thread Vladimir Davydov
ine_cpus except they block not cpu, but memory hotplug. [ v1 can be found at https://lkml.org/lkml/2014/4/6/68. I NAK'ed it by myself, because it used an rw semaphore for get/put_online_mems, making them dead lock prune. ] Thanks, Vladimir Davydov (2): mem-hotplug: implement g

[PATCH -mm v2 2/2] slab: get_online_mems for kmem_cache_{create,destroy,shrink}

2014-04-07 Thread Vladimir Davydov
applied, there is no need in taking the slab_mutex for kmem_cache_shrink any more, so it is removed from there. Signed-off-by: Vladimir Davydov --- mm/slab.c| 26 ++ mm/slab.h|1 + mm/slab_common.c | 35 +-- mm/slob.

[PATCH -mm v2 1/2] mem-hotplug: implement get/put_online_mems

2014-04-07 Thread Vladimir Davydov
, etc. Signed-off-by: Vladimir Davydov --- include/linux/memory_hotplug.h | 14 ++-- include/linux/mmzone.h |8 +-- mm/kmemleak.c |4 +- mm/memory-failure.c|8 +-- mm/memory_hotplug.c| 142 mm/s

[PATCH TRIVIAL] memcg: un-export __memcg_kmem_get_cache

2014-04-07 Thread Vladimir Davydov
It is only used in slab and should not be used anywhere else so there is no need in exporting it. Signed-off-by: Vladimir Davydov --- mm/memcontrol.c |1 - 1 file changed, 1 deletion(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index b4b6aef562fa..874594f57c90 100644 --- a/mm

[PATCH] slub: fix memcg_propagate_slab_attrs

2014-04-07 Thread Vladimir Davydov
After creating a cache for a memcg we should initialize its sysfs attrs with the values from its parent. That's what memcg_propagate_slab_attrs is for. Currently it's broken - we clearly muddled root-vs-memcg caches there. Let's fix it up. Signed-off-by: Vladimir Davydov --- m

Re: [patch 1/3] timerfd: Implement show_fdinfo method

2014-04-07 Thread Vladimir Davydov
glx@): > > - Add it_value/it_interval fields > - Save flags being used in timerfd_setup in context > > CC: Shawn Landden > CC: Thomas Gleixner > CC: Andrew Morton > CC: Andrey Vagin > CC: Pavel Emelyanov > CC: Vladimir Davydov > Signed-off-by: Cyrill Gorcunov

Re: [patch 1/3] timerfd: Implement show_fdinfo method

2014-04-07 Thread Vladimir Davydov
On 04/08/2014 10:42 AM, Cyrill Gorcunov wrote: > On Tue, Apr 08, 2014 at 10:17:14AM +0400, Vladimir Davydov wrote: > ... >>> +static int timerfd_show(struct seq_file *m, struct file *file) >>> +{ >>> + struct timerfd_ctx *ctx = file->priva

Re: [patch 2/3] docs: procfs -- Document timerfd output, v2

2014-04-08 Thread Vladimir Davydov
On 04/08/2014 11:43 AM, Cyrill Gorcunov wrote: > CC: Shawn Landden > CC: Thomas Gleixner > CC: Andrew Morton > CC: Andrey Vagin > CC: Pavel Emelyanov > CC: Vladimir Davydov > Signed-off-by: Cyrill Gorcunov > --- > > Vladimir, would the below text be more clear?

[PATCH 1/5] mm: vmscan: shrink all slab objects if tight on memory

2014-01-11 Thread Vladimir Davydov
/cgroups/msg06913.html Signed-off-by: Vladimir Davydov Cc: Andrew Morton Cc: Mel Gorman Cc: Michal Hocko Cc: Johannes Weiner Cc: Rik van Riel Cc: Dave Chinner Cc: Glauber Costa --- mm/vmscan.c | 25 + 1 file changed, 21 insertions(+), 4 deletions(-) diff --git a/mm

[PATCH 4/5] mm: vmscan: move call to shrink_slab() to shrink_zones()

2014-01-11 Thread Vladimir Davydov
This reduces the indentation level of do_try_to_free_pages() and removes extra loop over all eligible zones counting the number of on-LRU pages. Signed-off-by: Vladimir Davydov Reviewed-by: Glauber Costa Cc: Andrew Morton Cc: Mel Gorman Cc: Michal Hocko Cc: Johannes Weiner Cc: Rik van Riel

[PATCH 5/5] mm: vmscan: remove shrink_control arg from do_try_to_free_pages()

2014-01-11 Thread Vladimir Davydov
let's move shrink_control initialization to shrink_zones(). Signed-off-by: Vladimir Davydov Cc: Andrew Morton Cc: Mel Gorman Cc: Michal Hocko Cc: Johannes Weiner Cc: Rik van Riel Cc: Dave Chinner Cc: Glauber Costa --- mm/vmscan.c | 32 1 file change

[PATCH 3/5] mm: vmscan: respect NUMA policy mask when shrinking slab on direct reclaim

2014-01-11 Thread Vladimir Davydov
from shrinking inode/dentry caches from other nodes, which is not good. Fix this. Signed-off-by: Vladimir Davydov Cc: Andrew Morton Cc: Mel Gorman Cc: Michal Hocko Cc: Johannes Weiner Cc: Rik van Riel Cc: Dave Chinner Cc: Glauber Costa --- mm/vmscan.c |4 ++-- 1 file changed, 2 insertions(

[PATCH 2/5] mm: vmscan: call NUMA-unaware shrinkers irrespective of nodemask

2014-01-11 Thread Vladimir Davydov
fix it. Signed-off-by: Vladimir Davydov Reported-by: Dave Chinner Cc: Andrew Morton Cc: Mel Gorman Cc: Michal Hocko Cc: Johannes Weiner Cc: Rik van Riel Cc: Glauber Costa --- mm/vmscan.c | 19 ++- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/mm/vmscan.c b

Re: [PATCH v11 00/15] kmemcg shrinkers

2013-11-25 Thread Vladimir Davydov
Hi, Thank you for the review. I agree with all your comments and I'll resend the fixed version soon. If anyone still has something to say about the patchset, I'd be glad to hear from them. On 11/25/2013 09:41 PM, Johannes Weiner wrote: I ran out of steam reviewing these because there were

Re: [Devel] [PATCH v11 00/15] kmemcg shrinkers

2013-11-26 Thread Vladimir Davydov
On 11/26/2013 10:47 AM, Vladimir Davydov wrote: Hi, Thank you for the review. I agree with all your comments and I'll resend the fixed version soon. If anyone still has something to say about the patchset, I'd be glad to hear from them. On 11/25/2013 09:41 PM, Johannes Weiner

Re: [PATCH v11 00/15] kmemcg shrinkers

2013-11-26 Thread Vladimir Davydov
On 11/27/2013 02:47 AM, Dave Chinner wrote: On Tue, Nov 26, 2013 at 10:47:00AM +0400, Vladimir Davydov wrote: Hi, Thank you for the review. I agree with all your comments and I'll resend the fixed version soon. If anyone still has something to say about the patchset, I'd be glad to

[PATCH] memcg: fix kmem_account_flags check in memcg_can_account_kmem()

2013-11-27 Thread Vladimir Davydov
-by: Vladimir Davydov Cc: Johannes Weiner Cc: Michal Hocko Cc: Balbir Singh Cc: KAMEZAWA Hiroyuki --- mm/memcontrol.c |3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index f1a0ae6..40efb9d 100644 --- a/mm/memcontrol.c +++ b/mm

[PATCH] memcg: make memcg_update_cache_sizes() static

2013-11-27 Thread Vladimir Davydov
This function is not used outside of memcontrol.c so make it static. Signed-off-by: Vladimir Davydov Cc: Johannes Weiner Cc: Michal Hocko Cc: Balbir Singh Cc: KAMEZAWA Hiroyuki --- mm/memcontrol.c |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/memcontrol.c b/mm

Re: [PATCH] memcg: fix kmem_account_flags check in memcg_can_account_kmem()

2013-11-29 Thread Vladimir Davydov
On 11/29/2013 01:45 PM, Michal Hocko wrote: On Wed 27-11-13 19:46:01, Vladimir Davydov wrote: We should start kmem accounting for a memory cgroup only after both its kmem limit is set (KMEM_ACCOUNTED_ACTIVE) and related call sites are patched (KMEM_ACCOUNTED_ACTIVATED). This should be vice

Re: [PATCH 4/5] mm: vmscan: move call to shrink_slab() to shrink_zones()

2014-01-13 Thread Vladimir Davydov
On 01/14/2014 03:13 AM, Andrew Morton wrote: > On Sat, 11 Jan 2014 16:36:34 +0400 Vladimir Davydov > wrote: > >> This reduces the indentation level of do_try_to_free_pages() and removes >> extra loop over all eligible zones counting the number of on-LRU pages. >

Re: [PATCH 3/5] mm: vmscan: respect NUMA policy mask when shrinking slab on direct reclaim

2014-01-13 Thread Vladimir Davydov
On 01/14/2014 03:11 AM, Andrew Morton wrote: > On Sat, 11 Jan 2014 16:36:33 +0400 Vladimir Davydov > wrote: > >> When direct reclaim is executed by a process bound to a set of NUMA >> nodes, we should scan only those nodes when possible, but currently we >> will scan

Re: [PATCH 1/5] mm: vmscan: shrink all slab objects if tight on memory

2014-01-13 Thread Vladimir Davydov
On 01/14/2014 03:05 AM, Andrew Morton wrote: > On Sat, 11 Jan 2014 16:36:31 +0400 Vladimir Davydov > wrote: > >> When reclaiming kmem, we currently don't scan slabs that have less than >> batch_size objects (see shrink_slab_node()): >> >>

Re: [PATCH 1/5] mm: vmscan: shrink all slab objects if tight on memory

2014-01-15 Thread Vladimir Davydov
On 01/15/2014 02:14 AM, Andrew Morton wrote: > On Tue, 14 Jan 2014 11:23:30 +0400 Vladimir Davydov > wrote: > >> On 01/14/2014 03:05 AM, Andrew Morton wrote: >>> That being said, I think I'll schedule this patch as-is for 3.14. Can >>> you please take a loo

Re: [PATCH 1/5] mm: vmscan: shrink all slab objects if tight on memory

2014-01-15 Thread Vladimir Davydov
On 01/15/2014 01:25 PM, Andrew Morton wrote: > On Wed, 15 Jan 2014 12:47:35 +0400 Vladimir Davydov > wrote: > >> On 01/15/2014 02:14 AM, Andrew Morton wrote: >>> On Tue, 14 Jan 2014 11:23:30 +0400 Vladimir Davydov >>> wrote: >>> >>>> O

[PATCH 1/2] e1000: fix lockdep warning in e1000_reset_task

2013-11-22 Thread Vladimir Davydov
sane delayed works instead of the pair timer+work to implement its delayed asynchronous handlers, and the e1000_down() synchronously cancels all the works so that the problem that commit 338c15 tried to cope with disappeared, and we don't need any locks in the handlers any more. Moreover, any lo

[PATCH 2/2] e1000: fix possible reset_task running after adapter down

2013-11-22 Thread Vladimir Davydov
moves cancel_delayed_work_sync(watchdog_task) at the beginning of e1000_down_and_stop() thus ensuring the race is impossible. Signed-off-by: Vladimir Davydov Cc: Tushar Dave Cc: Patrick McHardy Cc: David S. Miller --- drivers/net/ethernet/intel/e1000/e1000_main.c | 15 +++ 1 file

[PATCH v11 05/15] memcg: move stop and resume accounting functions

2013-11-25 Thread Vladimir Davydov
From: Glauber Costa I need to move this up a bit, and I am doing it in a separate patch just to reduce churn in the patch that needs it. Signed-off-by: Glauber Costa Cc: Johannes Weiner Cc: Michal Hocko Cc: Hugh Dickins Cc: Kamezawa Hiroyuki Cc: Andrew Morton --- mm/memcontrol.c | 62 ++

[PATCH v11 01/15] memcg: make cache index determination more robust

2013-11-25 Thread Vladimir Davydov
From: Glauber Costa I caught myself doing something like the following outside memcg core: memcg_id = -1; if (memcg && memcg_kmem_is_active(memcg)) memcg_id = memcg_cache_id(memcg); to be able to handle all possible memcgs in a sane manner. In particular, the roo

[PATCH v11 02/15] memcg: consolidate callers of memcg_cache_id

2013-11-25 Thread Vladimir Davydov
From: Glauber Costa Each caller of memcg_cache_id ends up sanitizing its parameters in its own way. Now that the memcg_cache_id itself is more robust, we can consolidate this. Also, as suggested by Michal, a special helper memcg_cache_idx is used when the result is expected to be used directly a

[PATCH v11 14/15] memcg: reap dead memcgs upon global memory pressure

2013-11-25 Thread Vladimir Davydov
From: Glauber Costa When we delete kmem-enabled memcgs, they can still be zombieing around for a while. The reason is that the objects may still be alive, and we won't be able to delete them at destruction time. The only entry point for that, though, are the shrinkers. The shrinker interface, ho

[PATCH v11 13/15] vmpressure: in-kernel notifications

2013-11-25 Thread Vladimir Davydov
From: Glauber Costa During the past weeks, it became clear to us that the shrinker interface we have right now works very well for some particular types of users, but not that well for others. The later are usually people interested in one-shot notifications, that were forced to adapt themselves

[PATCH v11 15/15] memcg: flush memcg items upon memcg destruction

2013-11-25 Thread Vladimir Davydov
From: Glauber Costa When a memcg is destroyed, it won't be imediately released until all objects are gone. This means that if a memcg is restarted with the very same workload - a very common case, the objects already cached won't be billed to the new memcg. This is mostly undesirable since a cont

[PATCH v11 11/15] super: make icache, dcache shrinkers memcg-aware

2013-11-25 Thread Vladimir Davydov
y and inode, which seems to be too costly. Signed-off-by: Vladimir Davydov Cc: Glauber Costa Cc: Dave Chinner Cc: Mel Gorman Cc: Rik van Riel Cc: Johannes Weiner Cc: Michal Hocko Cc: Hugh Dickins Cc: Kamezawa Hiroyuki Cc: Andrew Morton --- fs/dcache.c| 25 +++-

[PATCH v11 09/15] memcg,list_lru: add per-memcg LRU list infrastructure

2013-11-25 Thread Vladimir Davydov
pointer to the appropriate list_lru object from a memcg or a kmem ptr, which should be further operated with conventional list_lru methods. Signed-off-by: Vladimir Davydov Cc: Glauber Costa Cc: Dave Chinner Cc: Mel Gorman Cc: Rik van Riel Cc: Johannes Weiner Cc: Michal Hocko Cc: Hugh Dickins Cc

[PATCH v11 12/15] memcg: allow kmem limit to be resized down

2013-11-25 Thread Vladimir Davydov
From: Glauber Costa The userspace memory limit can be freely resized down. Upon attempt, reclaim will be called to flush the pages away until we either reach the limit we want or give up. It wasn't possible so far with the kmem limit, since we had no way to shrink the kmem buffers other than usi

[PATCH v11 10/15] memcg,list_lru: add function walking over all lists of a per-memcg LRU

2013-11-25 Thread Vladimir Davydov
calls list_lru_walk(), but shrink_dcache_sb(), which is going to be the only user of this function, does not need it. Signed-off-by: Vladimir Davydov Cc: Glauber Costa Cc: Dave Chinner Cc: Mel Gorman Cc: Rik van Riel Cc: Johannes Weiner Cc: Michal Hocko Cc: Hugh Dickins Cc: Kamezawa Hiroyuki Cc: A

[PATCH v11 08/15] vmscan: take at least one pass with shrinkers

2013-11-25 Thread Vladimir Davydov
From: Glauber Costa In very low free kernel memory situations, it may be the case that we have less objects to free than our initial batch size. If this is the case, it is better to shrink those, and open space for the new workload then to keep them and fail the new allocations. In particular, w

[PATCH v11 06/15] memcg: per-memcg kmem shrinking

2013-11-25 Thread Vladimir Davydov
From: Glauber Costa If the kernel limit is smaller than the user limit, we will have situations in which our allocations fail but freeing user pages will buy us nothing. In those, we would like to call a specialized memcg reclaimer that only frees kernel memory and leave the user memory alone. T

[PATCH v11 04/15] memcg: move initialization to memcg creation

2013-11-25 Thread Vladimir Davydov
From: Glauber Costa Those structures are only used for memcgs that are effectively using kmemcg. However, in a later patch I intend to use scan that list inconditionally (list empty meaning no kmem caches present), which simplifies the code a lot. So move the initialization to early kmem creatio

[PATCH v11 03/15] vmscan: also shrink slab in memcg pressure

2013-11-25 Thread Vladimir Davydov
From: Glauber Costa Without the surrounding infrastructure, this patch is a bit of a hammer: it will basically shrink objects from all memcgs under memcg pressure. At least, however, we will keep the scan limited to the shrinkers marked as per-memcg. Future patches will implement the in-shrinker

[PATCH v11 07/15] memcg: scan cache objects hierarchically

2013-11-25 Thread Vladimir Davydov
From: Glauber Costa When reaching shrink_slab, we should descent in children memcg searching for objects that could be shrunk. This is true even if the memcg does not have kmem limits on, since the kmem res_counter will also be billed against the user res_counter of the parent. It is possible th

[PATCH v11 00/15] kmemcg shrinkers

2013-11-25 Thread Vladimir Davydov
kernel notifications memcg: reap dead memcgs upon global memory pressure memcg: flush memcg items upon memcg destruction Vladimir Davydov (3): memcg,list_lru: add per-memcg LRU list infrastructure memcg,list_lru: add function walking over all lists of a per-memcg LRU super: make icac

Re: [Devel] [PATCH v11 00/15] kmemcg shrinkers

2013-11-01 Thread Vladimir Davydov
rhead introduced by the series is reasonable. Thanks. On 10/24/2013 04:04 PM, Vladimir Davydov wrote: This patchset implements targeted shrinking for memcg when kmem limits are present. So far, we've been accounting kernel objects but failing allocations when short of memory. This is because our o

[PATCH v12 03/18] memcg: move initialization to memcg creation

2013-12-02 Thread Vladimir Davydov
creation. Signed-off-by: Glauber Costa Signed-off-by: Vladimir Davydov Cc: Johannes Weiner Cc: Michal Hocko Cc: Balbir Singh Cc: KAMEZAWA Hiroyuki --- mm/memcontrol.c |4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 144cb4c..3a4e2f8

[PATCH v12 09/18] vmscan: shrink slab on memcg pressure

2013-12-02 Thread Vladimir Davydov
nr_deferred per-shrinker counter to avoid memory cgroup isolation issues. Ideally, this counter should be made per-memcg. Signed-off-by: Vladimir Davydov Cc: Johannes Weiner Cc: Michal Hocko Cc: Dave Chinner Cc: Andrew Morton Cc: Mel Gorman Cc: Rik van Riel Cc: Al Viro Cc: Balbir Singh Cc

[PATCH v12 02/18] memcg: consolidate callers of memcg_cache_id

2013-12-02 Thread Vladimir Davydov
as an array index to make sure we never accesses in a negative index. Signed-off-by: Glauber Costa Signed-off-by: Vladimir Davydov Cc: Johannes Weiner Cc: Michal Hocko Cc: Balbir Singh Cc: KAMEZAWA Hiroyuki --- mm/memcontrol.c | 49 + 1 file

[PATCH v12 08/18] vmscan: do_try_to_free_pages(): remove shrink_control argument

2013-12-02 Thread Vladimir Davydov
to shrink_zones(). So let's move shrink_control initialization to shrink_zones(). Signed-off-by: Vladimir Davydov Cc: Johannes Weiner Cc: Michal Hocko Cc: Andrew Morton Cc: Mel Gorman Cc: Rik van Riel --- mm/vmscan.c | 32 1 file changed, 12 inser

[PATCH v12 15/18] memcg: allow kmem limit to be resized down

2013-12-02 Thread Vladimir Davydov
give up. Signed-off-by: Glauber Costa Signed-off-by: Vladimir Davydov Cc: Johannes Weiner Cc: Michal Hocko Cc: Balbir Singh Cc: KAMEZAWA Hiroyuki --- mm/memcontrol.c | 43 ++- 1 file changed, 38 insertions(+), 5 deletions(-) diff --git a/mm/memcont

[PATCH v12 16/18] vmpressure: in-kernel notifications

2013-12-02 Thread Vladimir Davydov
From: Glauber Costa During the past weeks, it became clear to us that the shrinker interface we have right now works very well for some particular types of users, but not that well for others. The latter are usually people interested in one-shot notifications, that were forced to adapt themselves

[PATCH v12 13/18] memcg: per-memcg kmem shrinking

2013-12-02 Thread Vladimir Davydov
ndent, we have no option rather than failing all GFP_NOFS allocations when we are close to the kmem limit. The best thing we can do in such a situation is to spawn the reclaimer in a background process hoping next allocations will succeed. Signed-off-by: Vladimir Davydov Cc: Johannes Weiner Cc: M

[PATCH v12 10/18] memcg,list_lru: add per-memcg LRU list infrastructure

2013-12-02 Thread Vladimir Davydov
pointer to the appropriate list_lru object from a memcg or a kmem ptr, which should be further operated with conventional list_lru methods. Signed-off-by: Vladimir Davydov Cc: Johannes Weiner Cc: Michal Hocko Cc: Dave Chinner Cc: Andrew Morton Cc: Al Viro Cc: Balbir Singh Cc: KAMEZAWA Hiroyuki

Re: [PATCH v12 00/18] kmemcg shrinkers

2013-12-02 Thread Vladimir Davydov
is used as the iterator by shrink_slab() and as the target memcg by a particular memcg-aware shrinker. I would appreciate if you could look at the new version and share your attitude toward it. Thank you. On 12/02/2013 03:19 PM, Vladimir Davydov wrote: Hi, This is the 12th iteration of Glaub

[PATCH v12 11/18] memcg,list_lru: add function walking over all lists of a per-memcg LRU

2013-12-02 Thread Vladimir Davydov
calls list_lru_walk(), but shrink_dcache_sb(), which is going to be the only user of this function, does not need it. Signed-off-by: Vladimir Davydov Cc: Johannes Weiner Cc: Michal Hocko Cc: Dave Chinner Cc: Andrew Morton Cc: Al Viro Cc: Balbir Singh Cc: KAMEZAWA Hiroyuki --- include/linux/list_lru.h |

[PATCH v12 12/18] fs: make icache, dcache shrinkers memcg-aware

2013-12-02 Thread Vladimir Davydov
y and inode, which seems to be too costly. Signed-off-by: Vladimir Davydov Cc: Johannes Weiner Cc: Michal Hocko Cc: Dave Chinner Cc: Andrew Morton Cc: Al Viro Cc: Balbir Singh Cc: KAMEZAWA Hiroyuki --- fs/dcache.c| 25 +++-- fs/inode.c |

[PATCH v12 18/18] memcg: flush memcg items upon memcg destruction

2013-12-02 Thread Vladimir Davydov
at a memcg that goes away most likely indicates an isolated workload that is terminated. Signed-off-by: Glauber Costa Signed-off-by: Vladimir Davydov Cc: Johannes Weiner Cc: Michal Hocko Cc: Balbir Singh Cc: KAMEZAWA Hiroyuki --- mm/memcontrol.c | 17 + 1 file changed, 17

[PATCH v12 00/18] kmemcg shrinkers

2013-12-02 Thread Vladimir Davydov
ure: in-kernel notifications memcg: reap dead memcgs upon global memory pressure memcg: flush memcg items upon memcg destruction Vladimir Davydov (11): memcg: move several kmemcg functions upper fs: do not use destroy_super() in alloc_super() fail path vmscan: rename shrink_slab() args to m

[PATCH v12 17/18] memcg: reap dead memcgs upon global memory pressure

2013-12-02 Thread Vladimir Davydov
From: Glauber Costa When we delete kmem-enabled memcgs, they can still be zombieing around for a while. The reason is that the objects may still be alive, and we won't be able to delete them at destruction time. The only entry point for that, though, are the shrinkers. The shrinker interface, ho

[PATCH v12 14/18] vmscan: take at least one pass with shrinkers

2013-12-02 Thread Vladimir Davydov
the direct reclaim case for memcg. Although this same technique can be applied to other situations just as well, we will start conservative and apply it for that case, which is the one that matters the most. Signed-off-by: Vladimir Davydov Cc: Johannes Weiner Cc: Michal Hocko Cc: Andrew Morton

[PATCH v12 06/18] vmscan: rename shrink_slab() args to make it more generic

2013-12-02 Thread Vladimir Davydov
, we will have to make up phony values for nr_pages_scanned and lru_pages again when doing kmem-only reclaim for a memory cgroup, which is possible if the cgroup has its kmem limit less than the total memory limit. Signed-off-by: Vladimir Davydov Cc: Johannes Weiner Cc: Michal Hocko Cc: Andrew

[PATCH v12 04/18] memcg: move several kmemcg functions upper

2013-12-02 Thread Vladimir Davydov
. Signed-off-by: Vladimir Davydov Cc: Johannes Weiner Cc: Michal Hocko Cc: Balbir Singh Cc: KAMEZAWA Hiroyuki --- mm/memcontrol.c | 92 +++ 1 file changed, 46 insertions(+), 46 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c

[PATCH v12 07/18] vmscan: move call to shrink_slab() to shrink_zones()

2013-12-02 Thread Vladimir Davydov
This reduces the indentation level of do_try_to_free_pages() and removes extra loop over all eligible zones counting the number of on-LRU pages. Signed-off-by: Vladimir Davydov Cc: Johannes Weiner Cc: Michal Hocko Cc: Andrew Morton Cc: Mel Gorman Cc: Rik van Riel --- mm/vmscan.c | 57

[PATCH v12 01/18] memcg: make cache index determination more robust

2013-12-02 Thread Vladimir Davydov
_cache_id and make sure it always return a meaningful value. Signed-off-by: Glauber Costa Signed-off-by: Vladimir Davydov Cc: Johannes Weiner Cc: Michal Hocko Cc: Balbir Singh Cc: KAMEZAWA Hiroyuki --- mm/memcontrol.c |4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a

[PATCH v12 05/18] fs: do not use destroy_super() in alloc_super() fail path

2013-12-02 Thread Vladimir Davydov
operly initialized so let's inline appropriate snippets from destroy_super() to alloc_super() fail path instead of using the whole function there. Signed-off-by: Vladimir Davydov Cc: Al Viro --- fs/super.c |9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/fs/su

[PATCH] memcg: remove KMEM_ACCOUNTED_ACTIVATED

2013-12-02 Thread Vladimir Davydov
y, we don't use cgroup_lock in memcg_update_kmem_limit(), and static_key_slow_inc() is called under the set_limit_mutex, but the leftover from the above-mentioned commit is still here. Let's remove it. Signed-off-by: Vladimir Davydov Cc: Johannes Weiner Cc: Michal Hocko Cc: Balbir Singh Cc: KAMEZ

Re: [PATCH] memcg: remove KMEM_ACCOUNTED_ACTIVATED

2013-12-02 Thread Vladimir Davydov
On 12/02/2013 10:26 PM, Glauber Costa wrote: On Mon, Dec 2, 2013 at 10:15 PM, Michal Hocko wrote: [CCing Glauber - please do so in other posts for kmem related changes] On Mon 02-12-13 17:08:13, Vladimir Davydov wrote: The KMEM_ACCOUNTED_ACTIVATED was introduced by commit a8964b9b ("

Re: [PATCH] memcg: remove KMEM_ACCOUNTED_ACTIVATED

2013-12-02 Thread Vladimir Davydov
On 12/02/2013 10:15 PM, Michal Hocko wrote: [CCing Glauber - please do so in other posts for kmem related changes] On Mon 02-12-13 17:08:13, Vladimir Davydov wrote: The KMEM_ACCOUNTED_ACTIVATED was introduced by commit a8964b9b ("memcg: use static branches when code not in use") i

Re: [PATCH] memcg: remove KMEM_ACCOUNTED_ACTIVATED

2013-12-03 Thread Vladimir Davydov
On 12/03/2013 11:56 AM, Glauber Costa wrote: > On Mon, Dec 2, 2013 at 11:21 PM, Vladimir Davydov > wrote: >> On 12/02/2013 10:26 PM, Glauber Costa wrote: >>> On Mon, Dec 2, 2013 at 10:15 PM, Michal Hocko wrote: >>>> [CCing Glauber - please do so in oth

Re: [PATCH v12 05/18] fs: do not use destroy_super() in alloc_super() fail path

2013-12-03 Thread Vladimir Davydov
On 12/03/2013 01:00 PM, Dave Chinner wrote: > On Mon, Dec 02, 2013 at 03:19:40PM +0400, Vladimir Davydov wrote: >> Using destroy_super() in alloc_super() fail path is bad, because: >> >> * It will trigger WARN_ON(!list_empty(&s->s_mounts)) since s_mounts is >>

Re: [PATCH v12 06/18] vmscan: rename shrink_slab() args to make it more generic

2013-12-03 Thread Vladimir Davydov
On 12/03/2013 01:33 PM, Dave Chinner wrote: >> kmemcg reclaim is introduced, we will have to make up phony values for >> nr_pages_scanned and lru_pages again when doing kmem-only reclaim for a >> memory cgroup, which is possible if the cgroup has its kmem limit less >> than the total memory limit.

Re: [PATCH v12 09/18] vmscan: shrink slab on memcg pressure

2013-12-03 Thread Vladimir Davydov
On 12/03/2013 02:48 PM, Dave Chinner wrote: >> @@ -236,11 +236,17 @@ shrink_slab_node(struct shrink_control *shrinkctl, >> struct shrinker *shrinker, >> return 0; >> >> /* >> - * copy the current shrinker scan count into a local variable >> - * and zero it so that other

<    1   2   3   4   5   6   7   8   9   10   >