Re: mlockall() and small memory systems
So, some further investigation shows that on sparc (but not sparc64) the vmspace for userland processes is pretty huge - the t_mincore test already has a vmspace vm-vm_map.size value of 9118 pages before it even does the first mmap. Given the low resource limits cited upthread (~3500 locked pages) we are already beyound that when we try mlockall() (though I guess most pages are r/o and shared, i.e. libc, but it is not clear to me, how that is accounted against the resource limits). The values on sparc64 (with 8k pages) for comparision: 1767 pages vm_map.size, 190 of that resident. At the end of the test, there are exactly 128 additional pages (1895) and 745 are resident. Is there an easy way to not count the shared r/o pages? So I could do dynamic scaling of the test area size, or even skip the test completely if it is hopeless. And, more important: why are the sparc space *that* big? Martin
Re: mlockall() and small memory systems
On Thu, 24 May 2012, Martin Husemann wrote: On Wed, May 23, 2012 at 07:15:41PM +0100, David Laight wrote: What are the default ulimit values? Good point. Page size is 4k, with 32MB the limits are # ulimit -a time (-t seconds) unlimited file (-f blocks ) unlimited data (-d kbytes ) 65536 stack (-s kbytes ) 8192 coredump (-c blocks ) unlimited memory(-m kbytes ) 10664 locked memory (-l kbytes ) 3554 process (-p processes ) 160 nofiles (-n descriptors) 128 vmemory (-v kbytes ) unlimited sbsize(-b bytes ) unlimited With 64 MB ram they are: time (-t seconds) unlimited file (-f blocks ) unlimited data (-d kbytes ) 65536 stack (-s kbytes ) 8192 coredump (-c blocks ) unlimited memory(-m kbytes ) 42776 locked memory (-l kbytes ) 14258 process (-p processes ) 160 nofiles (-n descriptors) 128 vmemory (-v kbytes ) unlimited sbsize(-b bytes ) unlimited Locked memory sounds like what we run into - but shouldn't mlockall() return some error code if we exceed the limit? Not necessarily. If the code does mlockall() with the MCL_FUTURE flag then any *new* pages are also supposed to be locked into memory. So if the process address space at the time the mlockall() is done is smaller than the locked memory limit, the mlockall() should succeed. But later if the mmap() makes the total process address space exceed the mmap() call should return an error instead of succeeding. I think there may be a bug in mmap(). You should check to see if the mmap() causes the process to cross the locked memory limit. Eduardo
mlockall() and small memory systems
In the regular sparc test runs on qemu the emulated sparc machine only has 32MB of ram. In this setup the /usr/tests/lib/libc/sys/t_mincore test mincore_resid fails. If we allow qemu ot provide more memory, the test succeeds. The part of the test that fails in low memory environments is: An anonymous mmap of 128 pages size is created: addr = mmap(NULL, npgs * page, PROT_READ | PROT_WRITE, MAP_ANON | MAP_PRIVATE, -1, (off_t)0); Nothing in this mapping is touched, so nothing gets paged in/modified. Via mincore() the test now validates that 0 of the mapped pages are resident. So far, this works. Now a mlockall(MCL_CURRENT|MCL_FUTURE) call is done. The test now expects all pages in the above mapping to be resident and again checks via mincore(). This is the part where the test fails: some of the pages are not resident. I do believe this is not a bug in the sparc pmap. Am I wrong? Is this an MI bug? Or a bogus test? Martin
Re: mlockall() and small memory systems
I have not tried the test, but ntpd(8) on m68k machines (with less than 32MB) always complains mlockall() failed. --- Izumi Tsutsui
Re: mlockall() and small memory systems
On Thu, May 24, 2012 at 01:30:39AM +0900, Izumi Tsutsui wrote: I have not tried the test, but ntpd(8) on m68k machines (with less than 32MB) always complains mlockall() failed. Yeah, that is because libc already is too big or something. However, in this case the mlockall() suceeds. Martin
Re: mlockall() and small memory systems
On Wed, May 23, 2012 at 06:15:32PM +0200, Martin Husemann wrote: In the regular sparc test runs on qemu the emulated sparc machine only has 32MB of ram. In this setup the /usr/tests/lib/libc/sys/t_mincore test mincore_resid fails. If we allow qemu ot provide more memory, the test succeeds. What are the default ulimit values? I think they depend on the systems physical memory size. Even the ones that would actually be limited by the amount of swap. David -- David Laight: da...@l8s.co.uk
Re: mlockall() and small memory systems
On Wed, May 23, 2012 at 07:15:41PM +0100, David Laight wrote: What are the default ulimit values? Good point. Page size is 4k, with 32MB the limits are # ulimit -a time (-t seconds) unlimited file (-f blocks ) unlimited data (-d kbytes ) 65536 stack (-s kbytes ) 8192 coredump (-c blocks ) unlimited memory(-m kbytes ) 10664 locked memory (-l kbytes ) 3554 process (-p processes ) 160 nofiles (-n descriptors) 128 vmemory (-v kbytes ) unlimited sbsize(-b bytes ) unlimited With 64 MB ram they are: time (-t seconds) unlimited file (-f blocks ) unlimited data (-d kbytes ) 65536 stack (-s kbytes ) 8192 coredump (-c blocks ) unlimited memory(-m kbytes ) 42776 locked memory (-l kbytes ) 14258 process (-p processes ) 160 nofiles (-n descriptors) 128 vmemory (-v kbytes ) unlimited sbsize(-b bytes ) unlimited Locked memory sounds like what we run into - but shouldn't mlockall() return some error code if we exceed the limit? Btw: I can not do a anita install for sparc when running on -current amd64, the install stalls somewhere arbitrarily. It seems to work fine on netbsd-5 (see test results on babylon5). No idea what broke this. Martin
