Greg, > Hi Luis, > > On 10/30/2012 08:38 PM, Luis Alves wrote: >>> What you are seeing though is a classic case of memory fragmentation. >> >> Thats true. If I spam a lot of 'ls' with other commands between, it >> eventually works. (I guess at some point a memory chunk of 1024k >> becomes available). >> I've tried to rebuild the kernel using as memory allocator SLOB and >> SLUB, but the result seems to be the same as using SLAB. >>
I've tried those solutions too -- no good. I decided that was because those are kernel memory allocation policies, and have no effect on user memory allocation. Is that correct? >>> I would dig into the busybox ls code. Wanting to allocate a block of >>> 512k in a single chunk seems a bit crazy. >> >> I've tried with standalone 'ls' and the result is the same. >> It also fails wanting to allocate exacly 528384 (always the same value >> and only when ls'ing NFS mounts). >> (the mem alloc value is actually pretty rounded 512k + 4k) >> >> Could it be related to NFS kernel code? > > It could be. From the stack back trace: > > [<008446e8>] 0x8446e8 (inside <warn_alloc_failed>) > [<0084683a>] 0x84683a (inside <__alloc_pages_nodemask>) > [<00851e34>] 0x851e34 (inside <do_mmap_pgoff>) > [<008526c2>] 0x8526c2 <sys_old_mmap> > [<008526c2>] 0x8526c2 <sys_old_mmap> > [<0084e200>] 0x84e200 (inside <vm_mmap_pgoff>) > [<0085270a>] 0x85270a (inside <sys_old_mmap>) > > That looks like an mmap system call. And mmap is the underlying > allocator used by malloc() in uClibc (for non-MMU). So to me it > looks like a malloc somewhere in busybox. That is just a first > guess, it needs some further investigation. > > >> Don't know if it's relevant, but I'm using latest kernel sources from >> your git, gcc-4.7.2, running kernel from flash (XIP) and user binaries >> compiled with -msep-data (running from flash also). > > I wouldn't suspect anything different. But if you can try older kernel > versions you could confirm that. > > >> By the way, like Larry pointed out, is there a way to compile user >> binaries with shared flat libraries? >> Long time ago I've tried to enable UCLIBC_FORMAT_SHARED_FLAT in uClibc >> config, but then it was failing building (don't remember where/why it >> fails). > > I haven't used shared libraries in these setups for years. So I am > not sure what state they are in. Seems like it might have bit rotted > from your experiences. > Rats. It seems to me the root of the problem on memory constrained systems is the power-of-2 memory allocator; a boxcar memory allocator would work better, I think. I have read there used to be a non-power-of-2 memory allocator, named page_alloc2. Is there any chance it can be revived for 2.6 kernels? (See my post of 23 October 2012.) > Regards > Greg > > >> On Tue, Oct 30, 2012 at 12:33 AM, Greg Ungerer <g...@snapgear.com> wrote: >>> Hi Luis, >>> >>> >>> On 30/10/12 05:38, Luis Alves wrote: >>>> >>>> Thanks for the explanation Larry. >>>> >>>> Meanwhile I've read some stuff about non-MMU memory alocation (I had >>>> the wrong idea of how memory was allocated). >>>> I guess my system ram is not that much (only 8Mb)... >>> >>> >>> What you are seeing though is a classic case of memory fragmentation. >>> You have eneough free RAM in total, but not enough in s ingle >>> contiguous block. On a paged MMU system this is not a problem, pages >>> can be mapped so you see a contiguous block the size you want. You >>> can't do that without no MMU and paging. >>> >>> >>> >>>> How would shared libs solve the problem? >>>> The problem is allocating memory for 'ls' data and not 'ls' itself. Is >>>> this correct? >>> >>> >>> I would dig into the busybox ls code. Wanting to allocate a block of >>> 512k in a single chunk seems a bit crazy. >>> >>> Regards >>> Greg >>> >>> >>> >>>> On Mon, Oct 29, 2012 at 5:49 PM, Larry Baker <ba...@usgs.gov> wrote: >>>>> >>>>> Luis, >>>>> >>>>> On 29 Oct 2012, at 8:32 AM, Luis Alves wrote: >>>>> >>>>> DMA: 36*4kB 30*8kB 29*16kB 21*32kB 6*64kB 7*128kB 4*256kB 3*512kB >>>>> 0*1024kB 0*2048kB 0*4096kB = 5360kB >>>>> >>>>> >>>>> The largest block of memory your system has available is 524288 >>>>> (512*1024). >>>>> >>>>> Allocation of length 528384 from process 94 (ls) failed >>>>> >>>>> >>>>> The requested size is larger than that, so uClinux tries to allocate the >>>>> next larger block size (1024K). There are none available. >>>>> >>>>> Any sugestion? >>>>> >>>>> >>>>> I do not know what determines how much memory ls allocates. >>>>> >>>>> I have similar memory allocation failures on my system. It does not >>>>> appear >>>>> to use a shared uclibc. I do not know (yet) how to enable building a >>>>> shared >>>>> uclibc or how to change the builds to use it. If the instructions are >>>>> straightforward, I would like to hear from someone. >>>>> >>>>> Regards, >>>>> Luis >>>>> _______________________________________________ >>>>> uClinux-dev mailing list >>>>> uClinux-dev@uclinux.org >>>>> http://mailman.uclinux.org/mailman/listinfo/uclinux-dev >>>>> This message was resent by uclinux-dev@uclinux.org >>>>> To unsubscribe see: >>>>> http://mailman.uclinux.org/mailman/options/uclinux-dev >>>>> >>>>> >>>>> Larry Baker >>>>> US Geological Survey >>>>> 650-329-5608 >>>>> ba...@usgs.gov >>>> >>>> _______________________________________________ >>>> uClinux-dev mailing list >>>> uClinux-dev@uclinux.org >>>> http://mailman.uclinux.org/mailman/listinfo/uclinux-dev >>>> This message was resent by uclinux-dev@uclinux.org >>>> To unsubscribe see: >>>> http://mailman.uclinux.org/mailman/options/uclinux-dev >>>> >>>> >>>> >>> >>> >>> -- >>> ------------------------------------------------------------------------ >>> Greg Ungerer -- Principal Engineer EMAIL: g...@snapgear.com >>> SnapGear Group, McAfee PHONE: +61 7 3435 2888 >>> 8 Gardner Close FAX: +61 7 3217 5323 >>> Milton, QLD, 4064, Australia WEB: http://www.SnapGear.com >> >> >> > > > -- > ------------------------------------------------------------------------ > Greg Ungerer -- Principal Engineer EMAIL: g...@snapgear.com > SnapGear Group, McAfee PHONE: +61 7 3435 2888 > 8 Gardner Close, FAX: +61 7 3891 3630 > Milton, QLD, 4064, Australia WEB: http://www.SnapGear.com Larry Baker US Geological Survey 650-329-5608 ba...@usgs.gov
_______________________________________________ uClinux-dev mailing list uClinux-dev@uclinux.org http://mailman.uclinux.org/mailman/listinfo/uclinux-dev This message was resent by uclinux-dev@uclinux.org To unsubscribe see: http://mailman.uclinux.org/mailman/options/uclinux-dev
