[Devel] Re: [PATCH] proc: check vma-vm_file before dereferencing

2012-10-16 Thread Stanislav Kinsbursky
16.10.2012 01:40, Andrew Morton пишет: On Mon, 15 Oct 2012 19:30:03 +0400 Stanislav Kinsbursky skinsbur...@parallels.com wrote: It can be equal to NULL. Please write better changelogs, so people do not have to ask questions such as: - Under what conditions does this bug trigger? - In

[Devel] Re: [PATCH v6 02/10] ipc: use key as id functionality for resource get system call introduced

2012-10-16 Thread Stanislav Kinsbursky
15.10.2012 23:39, Eric W. Biederman пишет: Stanislav Kinsbursky skinsbur...@parallels.com writes: This patch introduces new IPC resource get request flag IPC_PRESET, which should be interpreted as a request to try to allocate IPC slot with number, starting from value resented by key. IOW,

[Devel] Re: [PATCH v6 02/10] ipc: use key as id functionality for resource get system call introduced

2012-10-16 Thread Stanislav Kinsbursky
15.10.2012 23:47, Eric W. Biederman пишет: ebied...@xmission.com (Eric W. Biederman) writes: Stanislav Kinsbursky skinsbur...@parallels.com writes: This patch introduces new IPC resource get request flag IPC_PRESET, which should be interpreted as a request to try to allocate IPC slot with

[Devel] Re: [PATCH v6 04/10] ipc: add new SHM_SET command for sys_shmctl() call

2012-10-16 Thread Stanislav Kinsbursky
15.10.2012 23:00, Ben Hutchings пишет: On Mon, 2012-10-15 at 19:59 +0400, Stanislav Kinsbursky wrote: New SHM_SET command will be interpreted exactly as IPC_SET, but also will update key, cuid and cgid values. IOW, it allows to change existent key value. The fact, that key is not used is

[Devel] Re: [PATCH v6 06/10] glge driver: rename internal SEM_SET macro to SEM_INIT

2012-10-16 Thread Stanislav Kinsbursky
15.10.2012 20:34, Jitendra Kalsaria пишет: -Original Message- From: Stanislav Kinsbursky [mailto:skinsbur...@parallels.com] Sent: Monday, October 15, 2012 9:00 AM To: a...@linux-foundation.org Cc: catalin.mari...@arm.com; will.dea...@arm.com; dhowe...@redhat.com;

[Devel] Re: [PATCH v6 06/10] glge driver: rename internal SEM_SET macro to SEM_INIT

2012-10-16 Thread Stanislav Kinsbursky
15.10.2012 22:28, Ben Hutchings пишет: On Mon, 2012-10-15 at 20:00 +0400, Stanislav Kinsbursky wrote: The reason for shit patch is that SET_SET is desired to be a part of new part of API of IPC sys_semctl() system call. [...] Two spelling errors above. :-) Will fix, thanks. Ben. --

[Devel] Re: [PATCH v6 08/10] IPC: message queue receive cleanup

2012-10-16 Thread Stanislav Kinsbursky
16.10.2012 00:03, Ben Hutchings пишет: On Mon, 2012-10-15 at 20:00 +0400, Stanislav Kinsbursky wrote: This patch moves all message related manipulation into one function msg_fill(). Actually, two functions because of the compat one. Signed-off-by: Stanislav Kinsbursky skinsbur...@parallels.com

[Devel] Re: [PATCH v6 10/10] test: IPC message queue copy feture test

2012-10-16 Thread Stanislav Kinsbursky
15.10.2012 23:23, David Howells пишет: Stanislav Kinsbursky skinsbur...@parallels.com wrote: +CFLAGS += -I../../../../arch/x86/include/generated/ +CFLAGS += -I../../../../include/ +CFLAGS += -I../../../../usr/include/ +CFLAGS += -I../../../../arch/x86/include/ Do you need to add uapi/ into

[Devel] Re: [RFC PATCH] posix timers: allocate timer id per task

2012-10-16 Thread Stanislav Kinsbursky
15.10.2012 20:34, Eric Dumazet пишет: On Mon, 2012-10-15 at 20:17 +0400, Stanislav Kinsbursky wrote: This patch is required CRIU project (www.criu.org). To migrate processes with posix timers we have to make sure, that we can restore posix timer with proper id. Currently, this is not true,

[Devel] Re: [RFC PATCH] posix timers: allocate timer id per task

2012-10-16 Thread Stanislav Kinsbursky
15.10.2012 21:04, Peter Zijlstra пишет: On Mon, 2012-10-15 at 20:17 +0400, Stanislav Kinsbursky wrote: This patch is required CRIU project (www.criu.org). To migrate processes with posix timers we have to make sure, that we can restore posix timer with proper id. Currently, this is not true,

[Devel] Re: [PATCH v4 06/14] memcg: kmem controller infrastructure

2012-10-16 Thread Kamezawa Hiroyuki
(2012/10/12 18:13), Glauber Costa wrote: On 10/12/2012 12:57 PM, Michal Hocko wrote: On Fri 12-10-12 12:44:57, Glauber Costa wrote: On 10/12/2012 12:39 PM, Michal Hocko wrote: On Fri 12-10-12 11:45:46, Glauber Costa wrote: On 10/11/2012 04:42 PM, Michal Hocko wrote: On Mon 08-10-12

[Devel] Re: [RFC PATCH] posix timers: allocate timer id per task

2012-10-16 Thread Stanislav Kinsbursky
15.10.2012 23:08, Thomas Gleixner пишет: On Mon, 15 Oct 2012, Stanislav Kinsbursky wrote: This patch is required CRIU project (www.criu.org). To migrate processes with posix timers we have to make sure, that we can restore posix timer with proper id. Currently, this is not true, because timer

[Devel] Re: [PATCH v4 08/14] res_counter: return amount of charges after res_counter_uncharge

2012-10-16 Thread Kamezawa Hiroyuki
(2012/10/08 19:06), Glauber Costa wrote: It is useful to know how many charges are still left after a call to res_counter_uncharge. While it is possible to issue a res_counter_read after uncharge, this can be racy. If we need, for instance, to take some action when the counters drop down to

[Devel] Re: [PATCH v6 02/10] ipc: use key as id functionality for resource get system call introduced

2012-10-16 Thread Stanislav Kinsbursky
15.10.2012 23:39, Eric W. Biederman пишет: Changing the creator uid and creator gid and key is semantically ugly. BTW, changing the creator uid and creator gid is done already by IPC_SET call. So doing it in ipc_update_key() is redundant. I'll simplify. -- Best regards, Stanislav

[Devel] Re: [PATCH v4 09/14] memcg: kmem accounting lifecycle management

2012-10-16 Thread Kamezawa Hiroyuki
(2012/10/12 17:41), Michal Hocko wrote: On Fri 12-10-12 11:47:17, Glauber Costa wrote: On 10/11/2012 05:11 PM, Michal Hocko wrote: On Mon 08-10-12 14:06:15, Glauber Costa wrote: Because kmem charges can outlive the cgroup, we need to make sure that we won't free the memcg structure while

[Devel] Re: [PATCH v4 10/14] memcg: use static branches when code not in use

2012-10-16 Thread Kamezawa Hiroyuki
(2012/10/12 16:47), Glauber Costa wrote: On 10/11/2012 05:40 PM, Michal Hocko wrote: On Mon 08-10-12 14:06:16, Glauber Costa wrote: We can use static branches to patch the code in or out when not used. Because the _ACTIVE bit on kmem_accounted is only set after the increment is done, we

[Devel] Re: [PATCH v6 02/10] ipc: use key as id functionality for resource get system call introduced

2012-10-16 Thread Eric W. Biederman
Stanislav Kinsbursky skinsbur...@parallels.com writes: 15.10.2012 23:47, Eric W. Biederman пишет: Hmm. Come to think of it I don't see why you need to set the id at all. We are using an idr allocator which effectively offers the semantics that the lowest available id will be allocated. The

[Devel] [PATCH v5 12/14] execute the whole memcg freeing in free_worker

2012-10-16 Thread Glauber Costa
A lot of the initialization we do in mem_cgroup_create() is done with softirqs enabled. This include grabbing a css id, which holds ss-id_lock-rlock, and the per-zone trees, which holds rtpz-lock-rlock. All of those signal to the lockdep mechanism that those locks can be used in SOFTIRQ-ON-W

[Devel] [PATCH v5 02/14] memcg: Reclaim when more than one page needed.

2012-10-16 Thread Glauber Costa
From: Suleiman Souhlal ssouh...@freebsd.org mem_cgroup_do_charge() was written before kmem accounting, and expects three cases: being called for 1 page, being called for a stock of 32 pages, or being called for a hugepage. If we call for 2 or 3 pages (and both the stack and several slabs used in

[Devel] [PATCH v5 08/14] res_counter: return amount of charges after res_counter_uncharge

2012-10-16 Thread Glauber Costa
It is useful to know how many charges are still left after a call to res_counter_uncharge. While it is possible to issue a res_counter_read after uncharge, this can be racy. If we need, for instance, to take some action when the counters drop down to 0, only one of the callers should see it. This

[Devel] [PATCH v5 00/14] kmem controller for memcg.

2012-10-16 Thread Glauber Costa
Hi, This is the first part of the kernel memory controller for memcg. It has been discussed many times, and I consider this stable enough to be on tree. A follow up to this series are the patches to also track slab memory. They are not included here because I believe we could benefit from merging

[Devel] [PATCH v5 03/14] memcg: change defines to an enum

2012-10-16 Thread Glauber Costa
This is just a cleanup patch for clarity of expression. In earlier submissions, people asked it to be in a separate patch, so here it is. [ v2: use named enum as type throughout the file as well ] Signed-off-by: Glauber Costa glom...@parallels.com Acked-by: Kamezawa Hiroyuki

[Devel] [PATCH v5 01/14] memcg: Make it possible to use the stock for more than one page.

2012-10-16 Thread Glauber Costa
From: Suleiman Souhlal ssouh...@freebsd.org We currently have a percpu stock cache scheme that charges one page at a time from memcg-res, the user counter. When the kernel memory controller comes into play, we'll need to charge more than that. This is because kernel memory allocations will also

[Devel] [PATCH v5 04/14] kmem accounting basic infrastructure

2012-10-16 Thread Glauber Costa
This patch adds the basic infrastructure for the accounting of kernel memory. To control that, the following files are created: * memory.kmem.usage_in_bytes * memory.kmem.limit_in_bytes * memory.kmem.failcnt * memory.kmem.max_usage_in_bytes They have the same meaning of their user memory

[Devel] [PATCH v5 11/14] memcg: allow a memcg with kmem charges to be destructed.

2012-10-16 Thread Glauber Costa
Because the ultimate goal of the kmem tracking in memcg is to track slab pages as well, we can't guarantee that we'll always be able to point a page to a particular process, and migrate the charges along with it - since in the common case, a page will contain data belonging to multiple processes.

[Devel] [PATCH v5 06/14] memcg: kmem controller infrastructure

2012-10-16 Thread Glauber Costa
This patch introduces infrastructure for tracking kernel memory pages to a given memcg. This will happen whenever the caller includes the flag __GFP_KMEMCG flag, and the task belong to a memcg other than the root. In memcontrol.h those functions are wrapped in inline acessors. The idea is to

[Devel] [PATCH v5 09/14] memcg: kmem accounting lifecycle management

2012-10-16 Thread Glauber Costa
Because kmem charges can outlive the cgroup, we need to make sure that we won't free the memcg structure while charges are still in flight. For reviewing simplicity, the charge functions will issue mem_cgroup_get() at every charge, and mem_cgroup_put() at every uncharge. This can get expensive,

[Devel] [PATCH v5 10/14] memcg: use static branches when code not in use

2012-10-16 Thread Glauber Costa
We can use static branches to patch the code in or out when not used. Because the _ACTIVE bit on kmem_accounted is only set after the increment is done, we guarantee that the root memcg will always be selected for kmem charges until all call sites are patched (see memcg_kmem_enabled). This

[Devel] [PATCH v5 14/14] Add documentation about the kmem controller

2012-10-16 Thread Glauber Costa
Signed-off-by: Glauber Costa glom...@parallels.com CC: Frederic Weisbecker fweis...@redhat.com CC: Kamezawa Hiroyuki kamezawa.hir...@jp.fujitsu.com CC: Michal Hocko mho...@suse.cz CC: Christoph Lameter c...@linux.com CC: Pekka Enberg penb...@cs.helsinki.fi CC: Johannes Weiner han...@cmpxchg.org

[Devel] [PATCH v5 05/14] Add a __GFP_KMEMCG flag

2012-10-16 Thread Glauber Costa
This flag is used to indicate to the callees that this allocation is a kernel allocation in process context, and should be accounted to current's memcg. It takes numerical place of the of the recently removed __GFP_NO_KSWAPD. [ v4: make flag unconditional, also declare it in trace code ]

[Devel] [PATCH v5 13/14] protect architectures where THREAD_SIZE = PAGE_SIZE against fork bombs

2012-10-16 Thread Glauber Costa
Because those architectures will draw their stacks directly from the page allocator, rather than the slab cache, we can directly pass __GFP_KMEMCG flag, and issue the corresponding free_pages. This code path is taken when the architecture doesn't define CONFIG_ARCH_THREAD_INFO_ALLOCATOR (only

[Devel] [PATCH v5 07/14] mm: Allocate kernel pages to the right memcg

2012-10-16 Thread Glauber Costa
When a process tries to allocate a page with the __GFP_KMEMCG flag, the page allocator will call the corresponding memcg functions to validate the allocation. Tasks in the root memcg can always proceed. To avoid adding markers to the page - and a kmem flag that would necessarily follow, as much

[Devel] Re: [PATCH v6 02/10] ipc: use key as id functionality for resource get system call introduced

2012-10-16 Thread Stanislav Kinsbursky
16.10.2012 13:03, Eric W. Biederman пишет: Stanislav Kinsbursky skinsbur...@parallels.com writes: 15.10.2012 23:47, Eric W. Biederman пишет: Hmm. Come to think of it I don't see why you need to set the id at all. We are using an idr allocator which effectively offers the semantics that the

[Devel] Re: [PATCH v5 04/14] kmem accounting basic infrastructure

2012-10-16 Thread Michal Hocko
On Tue 16-10-12 14:16:41, Glauber Costa wrote: This patch adds the basic infrastructure for the accounting of kernel memory. To control that, the following files are created: * memory.kmem.usage_in_bytes * memory.kmem.limit_in_bytes * memory.kmem.failcnt *

[Devel] Re: [PATCH v5 05/14] Add a __GFP_KMEMCG flag

2012-10-16 Thread Michal Hocko
On Tue 16-10-12 14:16:42, Glauber Costa wrote: This flag is used to indicate to the callees that this allocation is a kernel allocation in process context, and should be accounted to current's memcg. It takes numerical place of the of the recently removed __GFP_NO_KSWAPD. [ v4: make flag

[Devel] Re: [PATCH v5 14/14] Add documentation about the kmem controller

2012-10-16 Thread Michal Hocko
On Tue 16-10-12 14:16:51, Glauber Costa wrote: Signed-off-by: Glauber Costa glom...@parallels.com CC: Frederic Weisbecker fweis...@redhat.com CC: Kamezawa Hiroyuki kamezawa.hir...@jp.fujitsu.com CC: Michal Hocko mho...@suse.cz CC: Christoph Lameter c...@linux.com CC: Pekka Enberg

[Devel] Re: [PATCH v5 07/14] mm: Allocate kernel pages to the right memcg

2012-10-16 Thread Christoph Lameter
On Tue, 16 Oct 2012, Glauber Costa wrote: To avoid adding markers to the page - and a kmem flag that would necessarily follow, as much as doing page_cgroup lookups for no reason, whoever is marking its allocations with __GFP_KMEMCG flag is responsible for telling the page allocator that this

[Devel] Re: [PATCH v2 00/11] fuse: optimize scatter-gather direct IO

2012-10-16 Thread Maxim V. Patlasov
Hi Miklos, 09/19/2012 08:31 PM, Maxim Patlasov пишет: Hi, Existing fuse implementation processes scatter-gather direct IO in suboptimal way: fuse_direct_IO passes iovec[] to fuse_loop_dio and the latter calls fuse_direct_read/write for each iovec from iovec[] array. Thus we have as many

[Devel] Re: [PATCH v5 14/14] Add documentation about the kmem controller

2012-10-16 Thread Christoph Lameter
On Tue, 16 Oct 2012, Glauber Costa wrote: + memory.kmem.limit_in_bytes # set/show hard limit for kernel memory + memory.kmem.usage_in_bytes # show current kernel memory allocation + memory.kmem.failcnt # show the number of kernel memory usage hits limits +

[Devel] Re: [PATCH v5 07/14] mm: Allocate kernel pages to the right memcg

2012-10-16 Thread Glauber Costa
On 10/16/2012 07:31 PM, Christoph Lameter wrote: On Tue, 16 Oct 2012, Glauber Costa wrote: To avoid adding markers to the page - and a kmem flag that would necessarily follow, as much as doing page_cgroup lookups for no reason, whoever is marking its allocations with __GFP_KMEMCG flag is

[Devel] Re: [PATCH v5 14/14] Add documentation about the kmem controller

2012-10-16 Thread Christoph Lameter
On Tue, 16 Oct 2012, Glauber Costa wrote: A limitation of kernel memory use would be good, for example, to prevent abuse from non-trusted containers in a high density, shared, container environment. But that would be against intentional abuse by someone who has code that causes the kernel to