On Thu, 31 May 2007 12:43:16 -0700 Herbert van den Bergh wrote: > Below is a patch I would like to submit for limiting process private > memory allocations via setrlimit(RLIMIT_DATA). > > Thanks, > Herbert van den Bergh. > > > This patch changes how the kernel limits memory allocations that are > controlled by setrlimit() using the RLIMIT_DATA resource. Currently the > kernel can limit the size of a process data, bss, and heap, but does not > limit the amount of process private memory that can be allocated with > the mmap() call. Since malloc() calls mmap() to allocate process memory > once brk() calls can no longer grow the heap, process private memory > allocations cannot be limited with RLIMIT_DATA. > > With this patch, not only memory in the data segment of a process, but > also private data mappings, both file-based and anonymous, are counted > toward the RLIMIT_DATA resource limit. Executable mappings, such as > text segments of shared objects, are not counted toward the private data > limit. The result is that malloc() will fail once the combined size of > the data segment and private data mappings reaches this limit. > > This brings the Linux behavior in line with what is documented in the > POSIX man page for setrlimit(3p). > > Signed-off-by: Herbert van den Bergh ([EMAIL PROTECTED]) > Signed-off-by: Dave McCracken ([EMAIL PROTECTED]) >
Howdy, The patch seems to have all tabs converted to spaces. This could be caused by thunderbird or by copy-and-pasting. Tbird needs help for sending patches without munging them. See http://mbligh.org/linuxdocs/Email/Clients/Thunderbird or google for "external editor" and use that with a decent text editor. Maybe it doesn't matter for this patch, but in general you should send patches made against the latest mainline kernel, such as 2.6.22-rc3 or 2.6.22-rc3-git4 or current git tree. There are a few lines that are > 80 characters. Please try to avoid that. ---------------------------------------------------------------------------------- > --- linux-2.6.21/fs/proc/task_mmu.c.orig 2007-05-29 21:05:31.000000000 > -0700 > +++ linux-2.6.21/fs/proc/task_mmu.c 2007-05-30 08:17:44.000000000 -0700 > @@ -31,7 +31,7 @@ > if (hiwater_rss < mm->hiwater_rss) > hiwater_rss = mm->hiwater_rss; > > - data = mm->total_vm - mm->shared_vm - mm->stack_vm; > + data = mm->total_vm - mm->shared_vm - mm->stack_vm - mm->exec_vm; > text = (PAGE_ALIGN(mm->end_code) - (mm->start_code & PAGE_MASK)) >> > 10; > lib = (mm->exec_vm << (PAGE_SHIFT-10)) - text; > buffer += sprintf(buffer, > --- linux-2.6.21/mm/mprotect.c.orig 2007-04-25 20:08:32.000000000 -0700 > +++ linux-2.6.21/mm/mprotect.c 2007-05-30 08:34:36.000000000 -0700 > @@ -162,6 +162,21 @@ > } > } > > + /* > + * Check against private data size limit. When execute permission > + * is removed, the mapping is counted toward the private data size. > + */ > + if (!(newflags & VM_EXEC) && (vma->vm_flags & VM_EXEC)) { > + unsigned long rlim = > current->signal->rlim[RLIMIT_DATA].rlim_cur; > + if (rlim < RLIM_INFINITY) { > + unsigned long priv_vm; > + priv_vm = mm->total_vm - mm->shared_vm - mm->exec_vm; > + priv_vm <<= PAGE_SHIFT; > + if (priv_vm + (end - start) > rlim) > + return -ENOMEM; > + } > + } > + > /* > * First try to merge with previous and/or next vma. > */ > --- linux-2.6.21/mm/mremap.c.orig 2007-04-25 20:08:32.000000000 -0700 > +++ linux-2.6.21/mm/mremap.c 2007-05-30 08:38:28.000000000 -0700 > @@ -343,6 +343,22 @@ > goto out; > } > > + /* > + * Check against private data size limit. Count memory allocations > + * which are not shared and not executable code as private data. > + */ > + if (!(vma->vm_flags & (VM_EXEC|VM_SHARED))) { > + unsigned long rlim = > current->signal->rlim[RLIMIT_DATA].rlim_cur; > + if (rlim < RLIM_INFINITY) { > + unsigned long priv_vm; > + struct mm_struct *mm = current->mm; > + priv_vm = mm->total_vm - mm->shared_vm - mm->exec_vm; > + priv_vm <<= PAGE_SHIFT; > + if (priv_vm + (new_len - old_len) > rlim) > + goto out; > + } > + } > + > if (vma->vm_flags & VM_ACCOUNT) { > charged = (new_len - old_len) >> PAGE_SHIFT; > if (security_vm_enough_memory(charged)) > --- linux-2.6.21/mm/mmap.c.orig 2007-05-29 21:05:58.000000000 -0700 > +++ linux-2.6.21/mm/mmap.c 2007-05-30 08:27:55.000000000 -0700 > @@ -245,16 +245,6 @@ > if (brk < mm->end_code) > goto out; > > - /* > - * Check against rlimit here. If this check is done later after the > test > - * of oldbrk with newbrk then it can escape the test and let the data > - * segment grow beyond its set limit the in case where the limit is > - * not page aligned -Ram Gupta > - */ > - rlim = current->signal->rlim[RLIMIT_DATA].rlim_cur; > - if (rlim < RLIM_INFINITY && brk - mm->start_data > rlim) > - goto out; > - > newbrk = PAGE_ALIGN(brk); > oldbrk = PAGE_ALIGN(mm->brk); > if (oldbrk == newbrk) > @@ -267,6 +257,18 @@ > goto out; > } > > + /* > + * Check against private data size limit. Count memory allocations > + * which are not shared and not executable code as private data. > + */ > + rlim = current->signal->rlim[RLIMIT_DATA].rlim_cur; > + if (rlim < RLIM_INFINITY) { > + unsigned long priv_vm; > + priv_vm = (mm->total_vm - mm->shared_vm - mm->exec_vm) << > PAGE_SHIFT; > + if (priv_vm + (newbrk-oldbrk) > rlim) > + goto out; > + } > + > /* Check against existing mmap mappings. */ > if (find_vma_intersection(mm, oldbrk, newbrk+PAGE_SIZE)) > goto out; > @@ -875,7 +877,8 @@ > = VM_STACK_FLAGS & (VM_GROWSUP|VM_GROWSDOWN); > > if (file) { > - mm->shared_vm += pages; > + if (flags & VM_SHARED) > + mm->shared_vm += pages; > if ((flags & (VM_EXEC|VM_WRITE)) == VM_EXEC) > mm->exec_vm += pages; > } else if (flags & stack_flags) > @@ -1041,6 +1044,20 @@ > if (!may_expand_vm(mm, len >> PAGE_SHIFT)) > return -ENOMEM; > > + /* > + * Check against private data size limit. Count memory allocations > + * which are not shared and not executable code as private data. > + */ > + if (!(vm_flags & (VM_SHARED|VM_EXEC))) { > + unsigned long rlim, priv_vm; > + rlim = current->signal->rlim[RLIMIT_DATA].rlim_cur; > + if (rlim < RLIM_INFINITY) { > + priv_vm = (mm->total_vm - mm->shared_vm - mm->exec_vm) << > PAGE_SHIFT; > + if (priv_vm + len > rlim) > + return -ENOMEM; > + } > + } > + > if (accountable && (!(flags & MAP_NORESERVE) || > sysctl_overcommit_memory == OVERCOMMIT_NEVER)) { > if (vm_flags & VM_SHARED) { > > > - --- ~Randy *** Remember to use Documentation/SubmitChecklist when testing your code *** - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/