Subject: Re: Re: [ast-developers] str="$( < bigfile )" with "bigfile" == 32MB triggers excessive memory usage... --------
> Hi! > > ---- > > Attached (as "bigfilecommandsubstitution001.sh.txt") is a short demo > which shows a problem in ast-ksh.2012-06-12 with memory usage for > str="$( < bigfile )" (and similar usage). The problem seems to be that > a larger input file size (for example 32MB) triggers excessive memory > usage, usually well beyond 512MB on Solaris 11/AMD64. > > Running the demo triggers the following error (this comes from the > limits defined via "ulimit" in the test (e.g. we assume that a file > size of 32MB will not cause more than 64MB of extra memory usage in > ksh93 (ksh93 itself is granted 16MB for itself&&running the script, > e.g. the ulimit limits are 80MB total in this case. However even that > is very very generous)): > -- snip -- > $ env - LC_ALL=C ~/bin/ksh /tmp/bigmem.sh > -rw-r--r-- 1 test001 users 33226753 Jun 21 08:10 mytestfile > bigfilecommandsubstitution001.sh: line 36: storage allocator out of > space on 8486912 byte request ( region 545751040 segments 133 busy > 151:8769120:8421280 free 139:536966640:8355744 ) [Not enough space] > -- snip -- > > > ---- > > Bye, > Roland > I have a patch that fixes this. However, a 32MB file will take more than 64MB since it needs to do the expansion, and then the assignment. However, it will not require 80MB total. David Korn [email protected] _______________________________________________ ast-developers mailing list [email protected] https://mailman.research.att.com/mailman/listinfo/ast-developers
