On Wed, Jan 26, 2005 at 12:07:50PM +0100, Ake wrote: > Use of rlim[RLIMIT_RSS] in mm/filemap.c is wrong. > It is passed down to kernel as a number of bytes but is being used as a > number of pages. > > There is also a misinformative comment in fs/proc/array.c > in proc_pid_stat where it says > mm ? mm->rss : 0, /* you might want to shift this left 3 */ > the number 3 should probably be PAGE_SHIFT-10.
Amazing that this has never been noticed before - I bet not many people use RSS limits with madvise(). This transform the rlimit in pages before the comparison, can you please test it. --- a/mm/filemap.c.orig 2004-11-17 09:54:22.000000000 -0200 +++ b/mm/filemap.c 2005-01-26 15:21:10.614842296 -0200 @@ -2609,6 +2609,9 @@ error = -EIO; rlim_rss = current->rlim ? current->rlim[RLIMIT_RSS].rlim_cur : LONG_MAX; /* default: see resource.h */ + + rlim_rss = (rlim_rss & PAGE_MASK) >> PAGE_SHIFT; + if ((vma->vm_mm->rss + (end - start)) > rlim_rss) return error; - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/